mgr/dashboard: run-frontend-unittests: fix CEPH_ROOT initialization

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend: JWT authentication implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: run-backend-api-request.sh: support for JTW tokens

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: backend: JWT based authentication

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #24544 from votdev/bug_24729

mgr/dashboard: Confirmation modal doesn't close

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24699 from rhcs-dashboard/add-tslint-rule

mgr/dashboard: Add TSLint rule "no-unused-variable"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24694 from tspmelo/wip-36544

mgr/dashboard: Fix problem with ErasureCodeProfileService

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24696 from dillaman/wip-36542

qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #24698 into master

* refs/pull/24698/head:
Merge PR #24697 into nautilus
ceph_test_msgr: fix authorizer behavior
Merge pull request #24667 from liewegas/wip-ec-thrash-full
Merge PR #24689 into nautilus
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24691 from iridescent-rsy/fix_typo

common: fix typos in BackoffThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24697 into nautilus

* refs/pull/24697/head:
ceph_test_msgr: fix authorizer behavior

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #24652 from badone/wip-undefined-gperftool-symbols-2

rpm: Use updated gperftools-libs at runtime

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

mgr/dashboard: Add TSLint rule
- TSlint no-unused-variable rule added.
- Cleanup: unused imports and variables.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #24591 from tspmelo/wip-testing-module

mgr/dashboard: Unit Tests cleanup

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

ceph_test_msgr: fix authorizer behavior

Fixes breakage from this PR 2152d8ffb73a507a3d08d48b38c5a8e73f887138.

Fixes: http://tracker.ceph.com/issues/36495
Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/qemu: use unique clone directory to avoid race with workunit

If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24667 from liewegas/wip-ec-thrash-full

qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23103 from ifed01/wip-ifed-bluefs-migrate

os/bluestore: allow ceph-bluestore-tool to coalesce, add and migrate BlueFS backing volumes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23090 from mingshuaiwang/master

OSD: ceph-osd parent process need to restart log service after fork

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #24689 into nautilus

* refs/pull/24689/head:
qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24689 into master

* refs/pull/24689/head:
qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
Merge PR #24625 into nautilus
qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Confirmation modal doesn't close

Fixes: https://tracker.ceph.com/issues/24729
Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix problem with ErasureCodeProfileService

ErasureCodeProfileService was being provided twice and that was causing
problems in production mode.

Fixes: https://tracker.ceph.com/issues/36544
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20004 from mogeb/steady-clock-tools-rados

librados: use steady clock for rados_mon_op_timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24658 from tchaikov/wip-18202-rebased

blkdev: Rework API and add FreeBSD support

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

common: fix typos in BackoffThrottle

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change

Fixes 95b7d2340c04dc7cf90085c89606b8c85a8f2803

Fixes: http://tracker.ceph.com/issues/36485
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24184 into master

* refs/pull/24184/head:
mgr/DaemonServer: remove any upmaps on merging PGs
mgr/DaemonServer: prevent merge if either pg is remapped|upmap
mgr/DaemonServer: move pending merge check for more consistent code
qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
mgr/DaemonServer: add option to bypass careful throttling for thrasher
PendingReleaseNotes: note about mgr/balancer/max_misplaced change
mgr/DaemonServer: remove stale/misleading check
mgr/DaemonServer: throttle pgp_num changes based on misplaced %
mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
mon/PGMap: add get_recovery_stats()
mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
pybind/mgr/mgr_module: add get_option()
mgr/DaemonServer: allow pg_num increases that abort pending merges
mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
mon/OSDMonitor: make pgp_num track pg_num more consistently

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #24654 into master

* refs/pull/24654/head:
osd: remove unused parameter 'dev' in OSD::mkfs function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: remove any upmaps on merging PGs

Remove any pg_upmap[_items] on pgs that are merging to ensure that they
land on the same OSDs.

This is a bit sloppy: we *could* set the source upmap to match the target
upmap (vs potentially moving both PGs to a third location, and/or then
having the balancer move the resulting PG somewhere else again), but for
now assume upmaps are not a common case and Keep It Simple.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: prevent merge if either pg is remapped|upmap

Remapping means they could be on different OSDs.

Fixes: http://tracker.ceph.com/issues/36166
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: move pending merge check for more consistent code

No functional change, but this makes the code simpler to read.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller

Thrash such that we still exercise the careful throttling in the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: add option to bypass careful throttling for thrasher

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20581 from chrone81/patch-1

doc: Fix EC k=3 m=2 profile overhead calculation example.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22349 from gregsfortytwo/wip-24368-osd-restarts

systemd: only restart 3 times in 30 minutes, as fast as possible

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24646 from dillaman/wip-36500

rbd-mirror: always attempt to restart canceled status update task

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24670 from falcon78921/wip-doc-multisite-link

doc: updated wording from federated to multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24595 from cbodley/wip-24505

rgw: user stats account for resharded buckets

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #24673 from tchaikov/wip-librbd-test-os-cleanup

librbd,test: remove unused context_cb() function, silence GCC warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24645 from cbodley/wip-36496

cls/user: cls_user_remove_bucket writes modified header

Merge PR #24611 into master

* refs/pull/24611/head:
mgr/diskprediction: Solve predictor model file not backward compatibility

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24623 into master

* refs/pull/24623/head:
mgr: Mutex::Locker -> std::lock_guard
kv: Mutex::Locker -> std::lock_guard
librados: Mutex::Locker -> std::lock_guard
mon: Mutex::Locker -> std::lock_guard
osd: Mutex::Locker -> std::lock_guard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #24605 from tchaikov/wip-use-ccache-the-old-way

cmake: fix compilation with distcc and other compiler wrappers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

blkdev: FreeBSD support

Signed-off-by: Alan Somers <asomers@gmail.com>

blkdev: overhaul API for better portability

* Turn the API into a class so it will work with GoogleMock
* Take file descriptors for all methods, instead of a mix of file
  descriptors, path names, and canonical device names.  It's more
  consistent and it will work better with FreeBSD.
* Split get_device_by_fd into block_device_partition and
  block_device_wholedisk
* Remove dead code

Signed-off-by: Alan Somers <asomers@gmail.com>

blkdev: move test_get_blkdev_size to test_get_blkdev_props

Also, enhance blkdev test program to show more properties than just size

Signed-off-by: Alan Somers <asomers@gmail.com>

blkdev: Add dedicated accessors for all property functions

Hide get_block_device_string_property and get_block_device_int_property
to lower risk of misuse.

Signed-off-by: Alan Somers <asomers@gmail.com>

blkdev: add an enum type for block device properties

Signed-off-by: Alan Somers <asomers@gmail.com>

Merge pull request #21155 from runsisi/wip-auth-cleanup

auth/cephx: minor code cleanup

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24672 from SUSE/wip-orchestrator-result-property

mgr/orchestrator: use result property in Completion classes

Merge pull request #24314 from rhcs-dashboard/dashboards

mgr/dashboard: Grafana dashboard updates and additions

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #24628 from Devp00l/wip-issue-36445

mgr/dashboard: Add missing test requirement "werkzeug"

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #24669 from falcon78921/wip-docs-36527

doc: add spacing to subcommand references

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19386 from ganeshmaharaj/qat-ceph-crypto

crypto: QAT based Encryption for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: remove unused context_cb() function

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/objectstore: silence "-Wsign-compare" warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/orchestrator: use result property in Completion classes

The _Completion class defined a get_result() method, but the orchestrator
CLI and Rook module both just expected a result property, so let's run with
the property version.

Signed-off-by: Tim Serong <tserong@suse.com>

auth/cephx: remove unused member method

Signed-off-by: runsisi <runsisi@zte.com.cn>

auth: remove unused forward declaration

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #24618 from CharlotteRose/wip-number-eviction-events

doc: numbered eviction situations

Reviewed-by: Jos Collin <jcollin@redhat.com>

crypto: Merge compilation of crypto accelerators under a single cmake.

- Control ISA-L and QAT under a crypto makefile
- Allow conditional compile of QAT.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

crypto: QAT Wrapper for Ceph Encryption

- Crypto plugin created and currently handles only AES_256_CBC
- Contents
-- Poller thread for QAT HW
-- Contiguous memory allocator using QAT's usdm driver
-- Do in-place crypto operation reducing mem usage.
-- Enable multi-QAT instance to allow parallel operations.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

doc: updated wording from federated to multisite

Updated the terminology from federated architecture
to multisite configuration. Updated the reference link for multisite
configuration.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

doc: add spacing to subcommand references

Added appropriate spacing for subcommand
code references.

Fixes: https://tracker.ceph.com/issues/36527
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

For EC pools we have a lot of shards, and 30% probability on each one
means we are very like to repeatedly fail backfill reservations.. long
enough that teuthology gives up waiting.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #24531 from alfredodeza/wip-rm36386

ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24453 from cbodley/wip-rgw-putobjprocessor

rgw: refactor PutObjProcessor stack

Merge pull request #23304 from smithfarm/wip-23776

 tools: ceph-authtool: report correct number of caps when creating keyring

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: numbered eviction situations

Signed-off-by: Scoots Hamilton <scoots@redhat.com>

Merge pull request #19571 from Fusl/patch-1

doc: doc/radosgw/s3.rst: Adding AWS S3 `Storage Class` as `Not Supported`

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24660 from tchaikov/wip-crimson-cleanup

crimson: crimson/mon: remove timeout support from mon::Client::authenticate()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24626 from jtlayton/wip-mgr-urlparse

mgr/rook: fix urljoin import

Merge pull request #24604 from yangdongsheng/token_fix

common/Throttle: TokenBucketThrottle: use reference to m_blockers.front()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24653 from mynaramana/claydoc

doc: erasure-code-clay fixes typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22162 from liuzhanhao/master

doc: Update cpp.rst to accommodate the new APIs in libs3

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: remove unused parameter 'dev' in OSD::mkfs function

Signed-off-by: Yao Guotao <yaoguot@gmail.com>

doc: erasure-code-clay fixes typos

Signed-off-by: Myna <mynaramana@gmail.com>

rpm: Use updated gperftools-libs at runtime

Due to ABI breakage in libtcmalloc.so.4 we need to specify the minimum
version to be used at runtime to be greater than or equal to the version
used at build time.

Fixes: http://tracker.ceph.com/issues/36508
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

tests/ceph-authtool: add reproducer for "with 0 caps" issue

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: ceph-authtool: report correct number of caps when creating

Fixes: https://tracker.ceph.com/issues/23776
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd-mirror: always attempt to restart canceled status update task

Fixes: http://tracker.ceph.com/issues/36500
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tool/bluestore-tool: print target path for bluestore symlinks.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

tests: add qa test case for bluefs volume coalescence

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: force multiple directories usage for bluefs no matter what
amount of volumes is configured.

This allows both coalescence and split for BlueFS backing volumes.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: introduce offline DB/WAL volume migration for
ceph-bluestore-tool.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

cls/user: cls_user_remove_bucket writes modified header

if cls_user_remove_bucket() modifies the header with dec_header_stats(),
it should write the updated header back

Fixes: http://tracker.ceph.com/issues/36496
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24503 into master

* refs/pull/24503/head:
qa: increase timeout for cleanup

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #24455 into master

* refs/pull/24455/head:
qa: use timeout for fs asok operations

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #24615 into master

* refs/pull/24615/head:
cmake: mark gcc 7 as minimum C++ compiler (due to std++17)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24617 from p-na/fix-python3-issue

mgr/dashboard: Fix Python3 issue

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

crimson/mon: remove timeout support from mon::Client::authenticate()

as the timeout parameter is not used by OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: reuse common_mountcephfs_objs in crimson-common

so we won't built them twice.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #24102 into master

* refs/pull/24102/head:
global: fix err_to_stderr can't work when it was configured to false
osd: replace cout/derr with dout(0) to avoid writing trivial logs to syslog file

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24236 into master

* refs/pull/24236/head:
osd: Log error and abort if fails to pick address

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #24359 into master

* refs/pull/24359/head:
qa/tests: update ansible version to 2.6 for master branch testing.
qa/tests: use lvm as default for ceph-ansible testing, this should also work with raw devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #24603 into master

* refs/pull/24603/head:
crush: get "ceph osd crush class create/rm" back

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24613 into master

* refs/pull/24613/head:
OSD: fix a typo err when increase recvoery bytes statistics

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24586 from mynaramana/claydoc

erasure-code: fixes alignment issue when clay code is used with jerasure, cauchy_orig

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24625 into nautilus

* refs/pull/24625/head:
qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>