mgr: summarize mon metrics in MMgrReport and sent it

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mon: send health_metrics to mgr

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mgr,osd: make osd_metric more popular

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

mon: add admin command to dump historic/slow ops

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

Merge pull request #20777 from wjwithagen/wjw-mgr_dashboard_test

test: only test dashboard_v2 when it is enabled

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20849 from ceph/wip-cv-env-flags

ceph-volume tests add optional flags for vagrant

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests add optional flags for vagrant

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20827 from jdurgin/wip-pg-log-trim-errors

osd: treat successful and erroroneous writes the same for log trimming

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20444 from cbodley/wip-qa-ssl

qa/rgw: verify suite tests civetweb with ssl

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20823 from rjfd/wip-dashboard-pools

mgr/dashboard_v2: Pool controller

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: support test task as CLI arg in rub-backend-api-tests.sh

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Pool controller implementation and tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20821 from tspmelo/fix-configuration-page

mgr/dashboard_v2: fix cluster configuration page

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #20806 from Devp00l/table-column-toggle

mgr/dashboard_v2: Add toggle able columns

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #20797 from rjfd/wip-dashboard-log-requests

mgr/dashboard: HTTP request logging

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20765 from sebastian-philipp/dashboard_v2-no-detail_route

mgr/dashboard_v2: Removed unused `tools.detail_route()`

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20772 from votdev/fix_osd_details

mgr/dashboard v2: Don't show details if multiple OSDs are selected

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #20809 from idryomov/wip-librbd-discard-create-truncate

librbd: create+truncate for whole-object layered discards

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20243 from ceph/wip-cd-volume-entry

qa/tests: add openstack volume info + lvs for ceph-volume

Merge pull request #20826 from liewegas/wip-fix-mstart-config

vstart.sh: fix mstart variables

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20819 from ygtzf/wip-install-deps

install-deps.sh: fix an error condition expression

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20419 from tchaikov/wip-22748

common/config: add units to options

Reviewed-by: John Spray <john.spray@redhat.com>

PrimaryLogPG: only trim up to osd_pg_log_trim_max entries at once

This prevents the fix for http://tracker.ceph.com/issues/22050 or
potential future bugs from causing too much latency by trimming too
many log entries at once.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: use proper variables for ceph.conf and dev dir

Signed-off-by: Sage Weil <sage@redhat.com>

PG, PrimaryLogPG: trim log and rollback info for error log entries

Regular updates piggyback some osd state for this purpose with
MOSDRepOp[Reply]. Do the same thing for pure log entry updates (write
errors and lost/revert additions) via MOSDPGUpdateLogMissing[Reply].

Fixes: http://tracker.ceph.com/issues/22050
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

qa/rgw: verify suite tests civetweb with ssl

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: loop over clients instead of config keys

we only run radosgw instances on clients that are specified directly in
the rgw task - this list omits entries that are brought in via overrides

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: s3test task overrides REQUESTS_CA_BUNDLE to use system certs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: s3test task conditionally sets is_secure

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: ragweed task conditionally sets is_secure

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: swift task conditionally sets auth_ssl

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add ssl certificate option for rgw frontend config

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add class for rgw endpoints

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa: add task to generate ssl certificates

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20805 from Devp00l/refresh-rotate

mgr/dashboard_v2: Rotate the refresh icon on load

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20788 from xiexingguo/wip-balancer-04

pybind/mgr/balancer: two more fixes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20791 from guzhongyan/fix-pycmd

pybind/ceph_argparse: accept flexible req

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: refactor pool info retrieval to service class

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Add toggle able columns

It's now possible to toggle columns on and off in the data table.

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #20787 from ceph/wip-rm23260

ceph-volume fix filestore OSD creation after mon-config changes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20813 from smithfarm/wip-make-dist-suse

Revert "make-dist: add OBS-specific release suffix on SUSE"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20682 from dillaman/wip-22960

librbd: abstract hard-coded journal and cache hooks on IO path

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20815 from capri1989/get-packages-luminous-update

doc: Updated the get-packages.rst to luminous

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard_v2: fix cluster configuration page

Configuration level 'develop' was changed to 'dev' in
1717d4c578be108d5b2c651305941974b2201b0f.
Because of this all the dev configurations were not showing.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: HTTP request logging

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard v2: Don't show details if multiple OSDs are selected

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20747 from votdev/add_required_directive

mgr/dashboard v2: Add CSS class for required form fields

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: fix an error condition expression

Signed-off-by: Yao Guotao <yaoguot@gmail.com>

librbd: create+truncate for whole-object layered discards

A whole-object layered discard is implemented as a truncate rather
than a delete: a dummy object is needed to prevent the CoW machinery
from kicking in.  However, a truncate on a non-existent object is
a no-op.  If the object doesn't exist in HEAD, a discard request is
effectively ignored -- parent blocks are still seen afterwards.

A non-exclusive create on an existing object is also a no-op, so the
fix is to do a compound create+truncate instead.

Fixes: http://tracker.ceph.com/issues/23285
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard_v2: Refactored `RESTControllerTest`:

* Made `RESTControllerTest` inherent from `ControllerTestCase`
* Refactored `ControllerTestCase`
* Simplified all tests that inherent from  `ControllerTestCase`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Removed unused `tools.detail_route()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Updated the get-packages.rst to luminous

Signed-off-by: Kai Wagner <kwagner@suse.com>

common/options: consolidate Option::to_str() with operator<<

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: add a simple test for conf_get() and conf_set()

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options,mon: switch some option to TYPE_SECS

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: specialize the settings for time duration

because of the ADL, boost::detail::variant::printer is not able to find
operator<<(ostream&, chrono::seconds) even the latter is defined. so we
need to define our own printer. and print_value() function, so it can be
used out of the compilation unit.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some cephfs settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some rbd settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some rgw options to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some osd settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some mon options to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: specialize the settings for size

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20745 from Liuchang0812/jsonify-pool-get

mon: invalid JSON returned when querying pool parameters

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20775 from tspmelo/fix-iscsi-unit-test

mgr/dashboard_v2: add mocked service provider for TcmuIscsiService

Reviewed-by: Volker Theile <vtheile@suse.com>

pybind/ceph_argparse: accept flexible req
True and true both acceptable

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #20778 from p-na/wip-mgr-fix-dashboard-backend-tests

mgr/dashboard: Fix backend tests for newer CherryPy versions

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20782 from sebastian-philipp/dashboard_v2_fix_config_options_endpoint

mgr/dashboard_v2: Fix test_cluster_configuration test

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20786 from dzafman/wip-zafman-log-trim

tools/ceph-objectstore-tool: command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert "make-dist: add OBS-specific release suffix on SUSE"

This reverts commit ca6c92b3ec2f715cf247e9c1009d315d9de9c55a.

The OBS-specific suffix potentially causes trouble when building outside of
OBS. Also, the OBS build process already includes a spec-file munging step
where this can be done.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: Add pg log trim command to ceph-objectstore-tool

Add test script that verifies the command in qa/standalone/osd

Fixes: http://tracker.ceph.com/issues/23242
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #20769 from oritwas/wip-rgw-20448

rgw: fix crash with rgw_run_sync_thread false

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

ceph-volume tests verify that mkfs filestore uses default flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare pass the cephx_secret for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.prepare use keyfile from stdin for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20590 from hitoshikamei/rbd-bench

rbd: fix thread_offsets calculation of rbd bench

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20689 from Songweibin/wip-23038

rbd/test: add snap protection test for ex/import

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20697 from Songweibin/wip-trash-rm

rbd: allow trash rm/purge when pool quota is full used

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20795 from Songweibin/wip-fix-trash

rbd: pool_percent_used should not divided by 100

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20759 from dzafman/wip-cleanup

test: Make clearer by moving code out of loop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20801 from theanalyst/dashboard-build-typo

cmake: typo fix when npm is not found

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

test/librbd: expect_object_may_exist(..., false) in DiscardRemoveTruncate

This is !is_no_op_for_nonexistent_object() test -- NONEXISTENT case is
the interesting one.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

test/librbd: clone with parent features in DiscardRemoveTruncate

... otherwise object map expectations aren't actually set.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard v2: Add CSS class for required form fields

Signed-off-by: Volker Theile <vtheile@suse.com>

dashboard: cmake: typo fix when npm is not found

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #20390 from theanalyst/rgw/openssl-init

rgw: setup locks for libopenssl

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
Reviewed-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #20649 from tianshan/rgw_datalog_list

rgw: datalog list support --shard-id and --marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20686 from fangyuxiangGL/mcc

rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20707 from Liuchang0812/rgw-es-plugin-urlencode

rgw: url_encode key name and instance in es sync module

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20756 from cbodley/wip-qa-rgw-verify-combine

qa/rgw: combine swift, s3tests, ragweed into single verify task

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

mgr/dashboard_v2: Rotate the refresh icon on load

The refresh icon of the data table will now rotate on loading events.

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #20767 from xiexingguo/wip-pg-delete

osd/OSD: batch-list objects to reduce memory consumption

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20764 from chenyupeng360/master

msg/async: size of EventCenter::file_events should be greater than fd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20740 from Songweibin/wip-skip-logical-devices

osd: skip dumping logical devices

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20743 from rjfd/wip-dashboard-settings

mgr/dashboard_v2: Configuration settings support

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20727 from linuxbox2/wip-clang-5

bluestore: define default value of LoglevelV only once (3 templates)

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20711 from guzhongyan/mgr-nodels

mon: 'node ls' mgr support

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20706 from xiexingguo/wip-balancer-03

pybind/mgr/balancer: fix pool-deletion vs auto-optimization race

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20704 from xiexingguo/wip-balancer-02

mon/OSDMonitor: kill pending upmap changes too if pool is gone

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20703 from xiexingguo/wip-balancer-01

osd/OSDMap: ignore PGs from pools of failure-domain OSD

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20644 from jcsp/wip-23166

mgr/prometheus: skip OSD output if missing from CRUSH devices

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>