mgr/dashboard: Add unit test case for controller/erasure_code_profile.py

Signed-off-by: Ranjitha G <ranjitha.kmg@gmail.com>

Merge pull request #24785 from falcon78921/wip-docs-36605

doc: purge subcommand link broken

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: used ceph osd command ref label

Referenced purge subcommand info via ceph osd command label.
Fixes: https://tracker.ceph.com/issues/36605
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #24743 from rzarzynski/wip-osd-avoid-osdmap-refcounting

 core: avoid unnecessary refcounting of OSDMap on OSD's hot paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24657 from xiexingguo/wip-rm-device-class-fix

mon/OSDMonitor: two "ceph osd crush class rm" fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24585 into master

* refs/pull/24585/head:
doc: add developer documentation on new cephfs reclaim interfaces

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21094 from guzhongyan/dne-cleanup

src: no 'dne' acronym in user cmd output

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22633 from dongbula/add-dbstatistics-for-filestore

OSD: add impl for filestore to get dbstatistics

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23528 from jcsp/wip-osdmon-application

mon: don't commit osdmap on no-op application ops

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #24504 from ErwanAliasr1/evelu-ceph-volume-choose_disk

Additional work on ceph-volume to add some choose_disk capabilities

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #24703 into master

* refs/pull/24703/head:
common: prefer std::size to ARRAY_SIZE macro

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24712 into master

* refs/pull/24712/head:
mgr/DaemonServer: "osd safe-to-destroy" - more verbose output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24713 into master

* refs/pull/24713/head:
mon: drop repeated 'goodchars' and add osd crush ls testcase

Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #24724 into master

* refs/pull/24724/head:
test/librados/aio: remove unused callbacks
test/librados/aio: wait for all completions properly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24736 into master

* refs/pull/24736/head:
os/tests: should read size_t options using get_val<Option::size_t>()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24767 from votdev/issue_36581

mgr/dashboard/qa: Fix various vstart_runner.py issues

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24584 from wido/docs-zabbix-template

doc: Update mgr/zabbix plugin documentation with link to Zabbix template

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24606 from p-na/osd-actions-fe

mgr/dashboard: Add support for managing individual OSD settings/characteristics in the frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

doc: Update mgr/zabbix plugin documentation with link to Zabbix template

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #24297 from ukernel/wip-36192

osdc: reduce ObjectCacher's memory fragments

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24756 from tspmelo/wip-contribs

.githubmap, .mailmap, .organizationmap: update contributors

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #24727 from zmc/wip-dashboard-gzip

mgr/dashboard: Enable gzip compression

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

.githubmap, .mailmap, .organizationmap: update contributors

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard/qa: CephfsTest - admin_socket() got an unexpected keyword argument 'timeout'

Adapt method arguments of LocalRemote::run() according to teuthology.orchestra.run.run() (see https://github.com/ceph/teuthology/blob/master/teuthology/orchestra/run.py#L364) to be able to run QA tests locally in a vstart cluster.

Fixes: http://tracker.ceph.com/issues/36581
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #24720 from gauravsitlani/master

doc: added rbd default features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: add developer documentation on new cephfs reclaim interfaces

Tracker: http://tracker.ceph.com/issues/36395
Signed-off-by: Jeff Layton <jlayton@redhat.com>

doc: added rbd default features

Fixes: https://tracker.ceph.com/issues/15000
Added RBD default features description

Signed-off-by: Gaurav Sitlani <gauravsitlani@riseup.net>

Merge PR #24716 into master

* refs/pull/24716/head:
test/log: drop redundant test case
common/StackStringStream: don't reserve before every insert

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24283 into master

* refs/pull/24283/head:
client: support for exporting multiple subdirectories in faked mode

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #24050 into master

* refs/pull/24050/head:
Revert "mds: adjust export size after renaming directory into freezing subtree"
mds: cleanup nested auth pins tracking

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24700 from cbodley/wip-27221

rgw: support server-side encryption when SSL is terminated in a proxy

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #24721 from cbodley/wip-librados-mono-timeout

librados: fix unitialized timeout in wait_for_osdmap

Reviewed-by: Mohamad Gebai <mgebai@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24680 from cbodley/wip-36537

rgw: fix vector index out of range in RGWReadDataSyncRecoveringShardsCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #24677 from cbodley/wip-rgw-beast-derr

rgw: beast frontend logs socket errors at level 4

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #24731 into master

* refs/pull/24731/head:
rados: fix admin/build-doc warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24641 from ricardoasmarques/fix-cloning-message

mgr/dashboard: Fix long running RBD cloning / copying message

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24587 from ceph/wip-rm36363

ceph-volume: adds a --prepare flag to `lvm batch`

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24750 from iotcg/master

doc: give pool name if default pool rbd is not created

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: give pool name if default pool rbd is not created

unless creating the default pool named to be "rbd",
it will fail without pointing the pool name when
using rbd command.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

ceph-volume: update man page for batch --prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

mgr/dashboard: Fix long running RBD cloning / copying message

Fixes: https://tracker.ceph.com/issues/36484
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #24693 from idryomov/wip-is-queued

msg/async: fix is_queued() semantics

Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add support for managing individual OSD settings/characteristics in the frontend

Fixes: http://tracker.ceph.com/issues/35448
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #24739 from tspmelo/wip-py3-itervalues

ceph.in: Add support for python 3

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23962 from aclamk/fio-fix-compilation

objectstore/test/fio: Fixed fio compilation when tcmalloc is used.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Enable gzip compression

This is related to http://tracker.ceph.com/issues/36453. It is far from
a complete solution, but seems like a positive move.

I tested this change by first disabling my browser cache, and then used
the /docs endpoint to query /api/dashboard/health. Before compression:
  Content-Length: 60748
  Time: 615ms
After:
  Content-Length: 7505
  Time: 92ms

Then, I logged into the dashboard as normal and reloaded the page once I
was in. Some values for the reload operation before compression:
  Total page load time: 58.48s
  vendor.js Content-Length: 6486025
  vendor.js time: 48.09s
After:
  Total page load time: 14.55s
  vendor.js Content-Length: 1143178
  vendor.js time: 4.50s

Signed-off-by: Zack Cerza <zack@redhat.com>

DashboardTestCase: add assertHeaders()

Signed-off-by: Zack Cerza <zack@redhat.com>

LocalCephManager.admin_socket: add timeout kwarg

This fixes "TypeError: admin_socket() got an unexpected keyword argument
'timeout'". The value is never used.

Signed-off-by: Zack Cerza <zack@redhat.com>

core: avoid get_epoch()-driven refcounting on OSDMap in ReplicatedBackend.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

core: avoid get_epoch()-driven refcounting on OSDMap in PrimaryLogPG.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph.in: Add support for python 3

This will allow the condition to work in both python 2 and 3 environments.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24738 from alfredodeza/wip-rm36492

ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #24665 into master

* refs/pull/24665/head:
common: remove unused ioprio thread/wq infrastructure

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge PR #24701 into master

* refs/pull/24701/head:
os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24707 into master

* refs/pull/24707/head:
build/ops: drop sgdisk build/runtime dependencies

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge PR #24687 into master

* refs/pull/24687/head:
src/test: fix unordered manifest-unset op

Reviewed-by: Sage Weil <sage@redhat.com>

rados: fix admin/build-doc warnings

Fixed admin/build-doc warnings in librados.h
and rados_types.h

Signed-off-by: Jos Collin <jcollin@redhat.com>

ceph-volume tests.api update monkeypatching to use **kw

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests allow **kw when monkeypatching

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume api.lvm: do not spit out errors on the terminal from LVM

Signed-off-by: Alfredo Deza <adeza@redhat.com>

os/tests: should read size_t options using get_val<Option::size_t>()

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24722 from zmc/rbd-redirect

mgr/dashboard: Redirect /block to /block/rbd

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Simplify frontend test helper

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: TableActionsComponent cleanup

Rename `buttonCondition` to `canBePrimary`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Rename `DeletionModalComponent` to `CriticalConfirmationModalComponent`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Cleanup OSD components

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

common: prefer std::size to ARRAY_SIZE macro

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

Merge pull request #19964 from runsisi/wip-build

cmake: move ceph-osdomap-tool, ceph-monstore-tool out of ceph-test

Reviewed-by: Kefu Chai <kchai@redhat.com>

librados: fix unitialized timeout in wait_for_osdmap

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24540 into master

* refs/pull/24540/head:
mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24709 into master

* refs/pull/24709/head:
PendingReleaseNotes: note about cephfs client state reclaim

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24466 into master

* refs/pull/24466/head:
common: assert if buffer advance length overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24529 into master

* refs/pull/24529/head:
mds: cleanup command descriptions

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #24640 into master

* refs/pull/24640/head:
mds: remove MonClient reconnect

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24533 into master

* refs/pull/24533/head:
qa: add timeouts for remote ops for client mounts

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #24679 from tspmelo/wip-36528

mgr/dashboard: Fix HttpClient Module imports in unit tests

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24632 from Devp00l/wip-issue-36468

mgr/dashboard: Handle class objects as regular objects in KV-table

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

test/librados/aio: remove unused callbacks

That callbacks are called is covered by many other tests.
These particular m_safe and m_complete fields were unused.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

test/librados/aio: wait for all completions properly

Using global semaphores was error-prone and unnecessary. Unbalanced
usage resulted in a race like:

aio_write object foo -> dropped on full OSD
aio_read object foo -> -ENOENT

The intent of the test was to wait for the write to complete, so
do that explicitly for each aio test and remove the semaphores.

Fixes: http://tracker.ceph.com/issues/24587
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24674 from sileht/master

ceph-volume: extract flake8 config

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Redirect /block to /block/rbd

Fixes: https://tracker.ceph.com/issues/36401
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #23242 from linuxbox2/wip-rgw-inflight-ops

rgw: set default objecter_inflight_ops = 24576

common: remove unused ioprio thread/wq infrastructure

This is all now obsolete, with no remaining users.

Move ceph_gettid into Thread.{cc,h}

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: do not remove device class still referenced by ec-profiles

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: make "ceph osd crush class rm" idempotent

Removing a non-existent device class should be generally okay.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24692 from tchaikov/wip-cmake-with-asan

cmake: fixes to enable WITH_ASAN with clang and GCC

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24664 from tspmelo/wip-36513

mgr/dashboard: Fix missing failed restore notification

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24637 from ricardoasmarques/fix-rbd-actions-disable

mgr/dashboard: Fix RBD actions disable

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24612 from Tinatinak/fix_bug24548

mgr/dashboard: Fixed documentation link on RGW page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix HttpClient Module imports in unit tests

Some of the tests were importing the wrong HttpClient Module and that was
causing random errors.

Fixes: https://tracker.ceph.com/issues/36528
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix missing failed restore notification

Fixes: http://tracker.ceph.com/issues/36513
Signed-off-by: Tiago Melo <tmelo@suse.com>

mon: drop repeated 'goodchars' and add osd crush ls testcase

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

mgr/DaemonServer: "osd safe-to-destroy" - more verbose output

For multiple inputs, we want to avoid a situation like:

```
$ ceph osd safe-to-destroy 0 1 2
Error EBUSY: OSD(s) 1,2 have 96 pgs currently mapped to them
$ ceph osd safe-to-destroy 0
Error EBUSY: OSD(s) 0 last reported they still store some PG data, and not all PGs are active+clean; we cannot be sure they aren't still needed.
```

The first command seems to be implying osd.0 is safe-to-destroy whereas it isn't.
This patch instead gives a complete report of each osd if multiple osds are specified.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24544 from votdev/bug_24729

mgr/dashboard: Confirmation modal doesn't close

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24699 from rhcs-dashboard/add-tslint-rule

mgr/dashboard: Add TSLint rule "no-unused-variable"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24694 from tspmelo/wip-36544

mgr/dashboard: Fix problem with ErasureCodeProfileService

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24696 from dillaman/wip-36542

qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Mykola Golub <mgolub@suse.com>

build/ops: drop sgdisk build/runtime dependencies

sgdisk was a dependency of ceph-disk, which has been ripped out.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #24698 into master

* refs/pull/24698/head:
Merge PR #24697 into nautilus
ceph_test_msgr: fix authorizer behavior
Merge pull request #24667 from liewegas/wip-ec-thrash-full
Merge PR #24689 into nautilus
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24691 from iridescent-rsy/fix_typo

common: fix typos in BackoffThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>