qa/workunits/rbd: test new move-to-trash on image remove config override

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: optionally move image to trash when removed by user

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move remove API method to api namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move trash API methods to api namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24407 from alfredodeza/wip-rm26953

ceph-volume: allow to specify --cluster-fsid instead of reading from ceph.conf

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24515 from tchaikov/wip-cmake-rgw

cmake: do not use plain target_link_libraries(rgw_a ...)

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23622 from renhwztetecs/renhw-wip-25103

mgr: fixup pgs show in unknown state

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24448 from votdev/bug_36325

mgr/dashboard: Performance counter progress bar keeps infinitely looping

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #23970 from noahdesu/mgr-always-on-mods-v2

mgr: always on modules v2

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24239 from votdev/python_ceph_audit_log

mgr: Add ability to trigger a cluster/audit log message from Python

Reviewed-by: John Spray <john.spray@redhat.com>

cmake: do not use plain target_link_libraries(rgw_a ...)

this addresses following error:

CMake Error at src/rgw/CMakeLists.txt:178 (target_link_libraries):
  The plain signature for target_link_libraries has already been used
with
  the target "rgw_a".  All uses of target_link_libraries with a target
must
  be either all-keyword or all-plain.

  The uses of the plain signature are here:

   * src/rgw/CMakeLists.txt:168 (target_link_libraries)

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24487 from badone/wip-undefined-references-to-dpdk-symbols

cmake/modules/BuildDPDK.cmake: Build required DPDK libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24450 from dillaman/wip-rbd-conf-keys

librbd: simplify config override handling

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24509 from badone/wip-releases-add-13.2.2

doc/releases: Add Mimic release 13.2.2

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24368 from tchaikov/wip-cmake-cleanup

cmake: use OpenSSL::Crypto instead of OPENSSL_LIBRARIES

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake/modules/BuildDPDK.cmake: Build required DPDK libraries

Make sure the mbuf and ethdev libraries are created as they export
required symbols.

Fixes: http://tracker.ceph.com/issues/36341
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

doc/releases: Add Mimic release 13.2.2

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge PR #24496 into master

* refs/pull/24496/head:
doc: fix typo in blkin.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24484 from hsiang41/mgr/predictor_model_file

mgr/diskprediction: Replace local predictor model file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21614 from Devp00l/pool-creation-form

mgr/dashboard: Pool management

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24497 from idryomov/wip-clarify-copy-from

include/rados: clarify which flags go where for copy_from

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

cmake: use OpenSSL::Crypto instead of OPENSSL_LIBRARIES

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/rados: clarify which flags go where for copy_from

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc: fix typo in blkin.rst

Signed-off-by: Scoots Hamilton <scoots@redhat.com>

Merge pull request #24491 from tchaikov/wip-add-erasure-clay-in-release-note

PendingReleaseNotes: note newly added CLAY code

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: New formatter byte unit tests

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Unset any compression setting for pools in UI

The dashboard API now can delete all compression options if the
compression mode is set to 'unset' during edit. Other dismissed options
will be overwritten with the reset value on submit, which will cause
the deletion of the previously set settings.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #23230 from s0nea/wip-dashboard-config-options-add

mgr/dashboard config options add

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Update PG update test

It is now commented out like it was before,
but I've added a comment what happened during this test with the QA
system. The problem was that even with only a increase of 1 PG the QA
cluster went into a cluster warning state and did not recover in time.
The QA coverage timeout is 2 minutes.

I could not reproduce this behavior with a local cluster, but I've
added a loop to wait until pgp and pg number are equal and the cluster
is in a healthy state again. This can take locally about 5 seconds.
The internal loop has a timeout of 3 minutes.

Fixes: https://tracker.ceph.com/issues/36362
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Unset compression arguments for pools

The dashboard backend can now unset all set compression arguments if the
compression mode is switched to 'unset'. In the case of 'unset' Ceph
itself will only delete the 'compression_mode' argument, not all other
set arguments. The other arguments that should be removed, too, are
added to the update arguments in order to delete all set arguments.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Readd pool notifications

As of the merge of cleaner notifications, the notification component was
refactored and move this caused a loss of previously added
notifications.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Use task list service in pool list

Now the pool list uses the task list service to monitor all tasks, in
order to extend each pool item with corresponding tasks if any. If there
is a running task for a pool you will see what action runs on it in the
listing.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Handle pool permissions

Handles pool action permissions in pool listing.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Pool delete

You can delete pools through the UI, this will generate a task that
deletes the pool in the background.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Pool create / edit

You can create/edit pools through the UI if you have the right
permissions.

You can specify the following:
* Name - can't be duplicated
* Type - replicated or erasure
* Crush rule set
  * Validates if you can use it
  * A popover tells which crush steps are used
* Replica size - depends on your selected rule and the amount of OSDs
* Erasure code profile
* PGs - will be recalculated on form changes (type, replica size,
    erasure profile, crush rule) only if not set before
* EC overwrites flag
* Compression - Algorithm / Min/max blob size / mode / ratio
* Application metadata - Predefined and custom applications as badges

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24471 from tchaikov/wip-doc-serve-doc

doc: add instructions about using "serve-doc" to preview built document

Reviewed-by: Alfredo Deza <adeza@redhat.com>

PendingReleaseNotes: note newly added CLAY code

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24477 from tchaikov/wip-live-with-slow-list-size

cmake: remove checking for GCC 5.1

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Return errors to subscriber of task-wrapper

Fixes: https://tracker.ceph.com/issues/36360
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Fixes user roles

Due to changes in the badges component, user roles had to be amended.

Fixes: https://tracker.ceph.com/issues/36357
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add badge filtering

It's now possible to filter badges and create badges via a create badges
label.

Fixes: https://tracker.ceph.com/issues/36357
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: add create functionality to the frontend

Add the missing functionality to create config options and related
tests to the frontend.

Fixes: http://tracker.ceph.com/issues/24455
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Enable custom badges

Enables custom badges within badges component.

It's possible to use custom validations and custom error messages.

Fixes: https://tracker.ceph.com/issues/36357
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Amend pool backend tasks

Amends pool tasks in order to get the right results back into the UI
and also to have tasks with the same attributes.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add pool type definition

Adds pool type definition which is used in pool listing.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add crush rule type definitions

There are two type definitions one for the crush rule itself and one
for the crush step definition which is used inside the crush rule
definition.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Erasure code profile interface

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: add missing parameter type and related test

Added missing parameter type and a test for this service method.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add config options form

Fixes: http://tracker.ceph.com/issues/24455
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add pattern validators

Add some pattern validators (ip, uuid, number, decimalNumber) that
are needed to validate the config option input values.

Fixes: http://tracker.ceph.com/issues/24455
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Erasure code service

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: add placeholder for filters to datatable

As (e.g.) the configuration option documentation has dropdown menus
with pre-defined filters we need a placeholder for them in the
datatable.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: move config table details to their own component

Fixes: http://tracker.ceph.com/issues/24455
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add backend method to create config options

Fixes: http://tracker.ceph.com/issues/24455
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: fix typo

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: fix missing import

Added the missing import otherwise the 'should create' test will fail
because the received component is undefined.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: refactor '_get_mon_allow_pool_delete_config'

Refactor '_get_mon_allow_pool_delete_config' method to be a little bit
more general. The method can now be used to get the value of every
config option known to the cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #24392 from joscollin/wip-fix-fs-status

pybind/mgr: identify invalid fs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24422 from mynaramana/claydoc

docs: add Clay code plugin documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23666 from rhcs-dashboard/grafana-integration

mgr/dashboard: Grafana graphs integration with dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Performance counter progress bar keeps infinitely looping

Fixes: https://tracker.ceph.com/issues/36325
Signed-off-by: Volker Theile <vtheile@suse.com>

README.md: bump up required cmake to 3.5.1

we started to use cmake 3.5.1 back in 34f1ee78

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: identify invalid fs

Identify invalid fs in 'fs status' output.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Replace local predictor model file

Replace local predictor model file for Debian package.
It is compatible with below library:
python-sklearn 0.19.1-3
python-joblib   0.11-1

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>

doc: clay code plugin

Signed-off-by: Myna <mynaramana@gmail.com>

ersaure-code: Cleanup of clay code source files.

* fixes the copyright on clay code source files

Signed-off-by: Myna <mynaramana@gmail.com>

Merge pull request #24457 from trociny/wip-qa-cli-namespace

qa/workunits/rbd: fix cli generic namespace test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: re-use pool config override helper method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: use pool config overrides for create state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: use pool config overrides for clone state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper method to apply pool-level config overrides

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid explicitly naming all RBD config options

Instead, only list the few that are excluded and use the config
schema to populate the remainder.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cmake: remove checking for GCC 5.1

this reverts bcc0511d3b857a8810493ca3d3f17ba053651af2,

because

* we already check for C++17 support in the same cmake list
* it's non trivial to check the O(1) std::list::size().
* we always build using the GCC 7.3 from SCLs on CentOS and RHEL. so the
  error message is always printed if we could have detected the O(1)
  std::list::size(). and we cannot fail the build if we are using an
  standard library with O(1) std::list::size().

so, this check is pointless.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24460 from falcon78921/wip-doc-36336

doc: updated cluster map reference link

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24369 from p-na/fix-osd-error

mgr/dashboard: Fix error when clicking on newly created OSD

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Adds dynamic documentation link

Fixes: https://tracker.ceph.com/issues/24999
Signed-off-by: Kanika Murarka <kmurarka@redhat.com>

Merge pull request #24326 from rhcs-dashboard/27050-landing-page-milestone-2

mgr/dashboard: New Landing Page: Milestone 2

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: add instructions about using "serve-doc" to preview built document

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix errors when clicking on new OSD

Fixes: http://tracker.ceph.com/issues/36245
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Add type hints to QA helper

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #24462 from falcon78921/wip-doc-36339

doc: updated crush map tunables link

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24416 from noahdesu/doc-req-no-clone

doc: install build-doc deps without git clone

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: updated crush map tunables link

Updated crush map tunables link in rados/configuration/mon-config-ref.rst
Fixes: https://tracker.ceph.com/issues/36339
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

doc: updated cluster map reference link

Updated cluster map reference link in rados/troubleshooting/troubleshooting-mon.rst
Fixes: https://tracker.ceph.com/issues/36336
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

qa/workunits/rbd: fix cli generic namespace test

expect_fail incorrectly unset '-e' option and if a consequent test
failed it did not abort the execution. And two typos in the namespace
tests were not detected due to this.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge PR #24437 into master

* refs/pull/24437/head:
PendingReleaseNotes: add note for mds drop cache

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #24444 from theanalyst/wip-reshard-stats

rgw: copy actual stats from the source shards during reshard

Merge pull request #23504 from pritha-srivastava/wip-sts-authentication

rgw: Code for STS Authentication.

librbd: remove non-performance sensitive cached config values

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: simplify filtering of config key overrides

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: use ConfigProxy within ImageCtx to store overrides

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/config: expose full config schema via the proxy

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Relocate cluster_log(). Only active modules can use it.

Signed-off-by: Volker Theile <vtheile@suse.com>

rgw: copy actual stats from the source shards during reshard

Currently we don't copy the actual_stats field during reshard, which makes
resharded buckets show a size_utilized as 0, which further has the problem that
a subsequent object removal would subtract the object size from the 0 size
utilized showing up large uint64_t values. Copy the size_actual from the source
object in both cls and in reshard_process. This will fix the new buckets,
existing buckets will still have to go through a bucket check --fix for their
stats to be corrected.

Fixes: http://tracker.ceph.com/issues/36290
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #24433 from dillaman/wip-rbd-mirror-doc

doc/rbd: explicitly state that mirroring requires connectivity to clusters

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24144 from tchaikov/wip-spdk

bluestore/NVMe: use PCIe selector as the path name

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24168 from tchaikov/wip-17516

cmake: detect armv8 crc and crypto feature using CHECK_C_COMPILER_FLAG

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24322 from votdev/improve_error_interceptor

mgr/dashboard: Improve error message handling

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

PendingReleaseNotes: add note for mds drop cache

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common/options: ensure all RBD-related integer options are unsigned

Signed-off-by: Jason Dillaman <dillaman@redhat.com>