mgr: fix MSG_MGR_MAP handling

ceph config show mgr.x doesn't work
root cause is mgr daemon's mgrc has no chance
to process MSG_MGR_MAP in the mgr daemon's
ms_dispatch.

fix is let this message pass through for mgrc

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
Signed-off-by: yupeng chen chenyupeng-it@360.cn

Merge PR #20700 into master

* refs/pull/20700/head:
client: use either dentry_invalidate_cb or remount_cb to invalidate kernel dcache

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #20719 into master

* refs/pull/20719/head:
client: add new ceph_mount_perms_set function

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #20780 into master

* refs/pull/20780/head:
doc/cephfs: mention "ceph fs set-default"
mds: include legacy_client_fscid in FSMap::dump

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20881 into master

* refs/pull/20881/head:
messages/MClientRequest: fix a typo so we don't output 'setfilelockrule' in logs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

messages/MClientRequest: fix a typo so we don't output 'setfilelockrule' in logs

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20864 from ceph/wip-cv-env-flags-fix

ceph-volume tests use granular env vars for vagrant

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20724 from mdw-at-linuxbox/wip-master-rgw-fixbuf

rgw: fix memory fragmentation problem reading data from client.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20872 from liewegas/wip-deterministic-cli-test

test/cli/osdmaptool/test-map-pgs.t: remove nondetermimistic test

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20739 from Jeegn-Chen/wip-reject-sse-mpcp

rgw: reject encrypted object COPY before supported

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

test/cli/osdmaptool/test-map-pgs.t: remove nondetermimistic test

This fails randomly; determinism is good!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20811 from tspmelo/fix-table-details

mgr/dashboard_v2: fix and improve table details

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20757 from tspmelo/wip-chart-tooltip

mgr/dashboard_v2: Improve charts tooltips

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

ceph-volume tests use granular env vars for vagrant

Fixes the error that would inject an extra argument to vagrant when the
environment variable is not present

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20830 from Songweibin/wip-cleanup

osd: two cleanups

Reviewed-by: Luo Kexue <luo.kexue@zte.com.cn>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20838 from badone/test_dump_pgstate_history-check-for-empty-output

tests: Check for empty output in test_dump_pgstate_history

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20839 from guzhongyan/tree-from-cleanup

mon/OSDMonitor: clean up cmd 'osd tree-from'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20831 from zhsj/fix-spelling

misc: fix various spelling errors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: iliul <liul.stone@gmail.com>

Merge pull request #20777 from wjwithagen/wjw-mgr_dashboard_test

test: only test dashboard_v2 when it is enabled

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20849 from ceph/wip-cv-env-flags

ceph-volume tests add optional flags for vagrant

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests add optional flags for vagrant

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20827 from jdurgin/wip-pg-log-trim-errors

osd: treat successful and erroroneous writes the same for log trimming

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard_v2: fix and improve table details

There was a problem with the current implementation of the table details, when
compiled in production mode, where the method that created the details element
would throw an exception.

Besides fixing that, this commit also improves the way we define the details.
Now instead of sending the component name, through the 'detailsComponent' attr,
we need to get the current selection via 'updateSelection' attr and declare the
detail component inside the cdTable and add the cdTableDetail attr to that
element.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: improve health page charts tooltips

Extracted the charts into a new component and it is now using a new tooltip.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: improve tooltip of cephfsChart

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: extract cephfs chart into a component

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: improve tooltip of cdSparkline

Added a new scss and class for the tooltip.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20444 from cbodley/wip-qa-ssl

qa/rgw: verify suite tests civetweb with ssl

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20823 from rjfd/wip-dashboard-pools

mgr/dashboard_v2: Pool controller

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: support test task as CLI arg in rub-backend-api-tests.sh

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Pool controller implementation and tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20821 from tspmelo/fix-configuration-page

mgr/dashboard_v2: fix cluster configuration page

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #20806 from Devp00l/table-column-toggle

mgr/dashboard_v2: Add toggle able columns

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #20797 from rjfd/wip-dashboard-log-requests

mgr/dashboard: HTTP request logging

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20765 from sebastian-philipp/dashboard_v2-no-detail_route

mgr/dashboard_v2: Removed unused `tools.detail_route()`

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20772 from votdev/fix_osd_details

mgr/dashboard v2: Don't show details if multiple OSDs are selected

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

qa/workunits/mon/crush_ops.sh: test osd tree-from

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

mon/OSDMonitor: clean up cmd 'osd tree-from'
Handle 'bucket not exist' and 'not a bucket' case

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

rgw: reject encrypted object COPY before supported

Current implementation does not follow S3 spec and even
may result in data corruption silently when copying
multipart objects accorss pools. So reject COPY operations
on encrypted objects before it is fully functional.

Fixes: http://tracker.ceph.com/issues/23232
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>

Merge pull request #20809 from idryomov/wip-librbd-discard-create-truncate

librbd: create+truncate for whole-object layered discards

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20243 from ceph/wip-cd-volume-entry

qa/tests: add openstack volume info + lvs for ceph-volume

misc: fix various spelling errors

Signed-off-by: Shengjing Zhu <i@zhsj.me>

Merge pull request #20826 from liewegas/wip-fix-mstart-config

vstart.sh: fix mstart variables

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osd: direct use of 'shard' without repeated calls

Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd: drop unused header in OSD.cc

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #20819 from ygtzf/wip-install-deps

install-deps.sh: fix an error condition expression

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20419 from tchaikov/wip-22748

common/config: add units to options

Reviewed-by: John Spray <john.spray@redhat.com>

PrimaryLogPG: only trim up to osd_pg_log_trim_max entries at once

This prevents the fix for http://tracker.ceph.com/issues/22050 or
potential future bugs from causing too much latency by trimming too
many log entries at once.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: use proper variables for ceph.conf and dev dir

Signed-off-by: Sage Weil <sage@redhat.com>

PG, PrimaryLogPG: trim log and rollback info for error log entries

Regular updates piggyback some osd state for this purpose with
MOSDRepOp[Reply]. Do the same thing for pure log entry updates (write
errors and lost/revert additions) via MOSDPGUpdateLogMissing[Reply].

Fixes: http://tracker.ceph.com/issues/22050
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

qa/rgw: verify suite tests civetweb with ssl

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: loop over clients instead of config keys

we only run radosgw instances on clients that are specified directly in
the rgw task - this list omits entries that are brought in via overrides

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: s3test task overrides REQUESTS_CA_BUNDLE to use system certs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: s3test task conditionally sets is_secure

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: ragweed task conditionally sets is_secure

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: swift task conditionally sets auth_ssl

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add ssl certificate option for rgw frontend config

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add class for rgw endpoints

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa: add task to generate ssl certificates

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20805 from Devp00l/refresh-rotate

mgr/dashboard_v2: Rotate the refresh icon on load

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20788 from xiexingguo/wip-balancer-04

pybind/mgr/balancer: two more fixes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20791 from guzhongyan/fix-pycmd

pybind/ceph_argparse: accept flexible req

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: refactor pool info retrieval to service class

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Add toggle able columns

It's now possible to toggle columns on and off in the data table.

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #20787 from ceph/wip-rm23260

ceph-volume fix filestore OSD creation after mon-config changes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20813 from smithfarm/wip-make-dist-suse

Revert "make-dist: add OBS-specific release suffix on SUSE"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20682 from dillaman/wip-22960

librbd: abstract hard-coded journal and cache hooks on IO path

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20815 from capri1989/get-packages-luminous-update

doc: Updated the get-packages.rst to luminous

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard_v2: fix cluster configuration page

Configuration level 'develop' was changed to 'dev' in
1717d4c578be108d5b2c651305941974b2201b0f.
Because of this all the dev configurations were not showing.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: HTTP request logging

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard v2: Don't show details if multiple OSDs are selected

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20747 from votdev/add_required_directive

mgr/dashboard v2: Add CSS class for required form fields

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: fix an error condition expression

Signed-off-by: Yao Guotao <yaoguot@gmail.com>

librbd: create+truncate for whole-object layered discards

A whole-object layered discard is implemented as a truncate rather
than a delete: a dummy object is needed to prevent the CoW machinery
from kicking in.  However, a truncate on a non-existent object is
a no-op.  If the object doesn't exist in HEAD, a discard request is
effectively ignored -- parent blocks are still seen afterwards.

A non-exclusive create on an existing object is also a no-op, so the
fix is to do a compound create+truncate instead.

Fixes: http://tracker.ceph.com/issues/23285
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard_v2: Refactored `RESTControllerTest`:

* Made `RESTControllerTest` inherent from `ControllerTestCase`
* Refactored `ControllerTestCase`
* Simplified all tests that inherent from  `ControllerTestCase`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Removed unused `tools.detail_route()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Updated the get-packages.rst to luminous

Signed-off-by: Kai Wagner <kwagner@suse.com>

common/options: consolidate Option::to_str() with operator<<

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: add a simple test for conf_get() and conf_set()

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options,mon: switch some option to TYPE_SECS

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: specialize the settings for time duration

because of the ADL, boost::detail::variant::printer is not able to find
operator<<(ostream&, chrono::seconds) even the latter is defined. so we
need to define our own printer. and print_value() function, so it can be
used out of the compilation unit.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some cephfs settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some rbd settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some rgw options to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some osd settings to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: switch some mon options to TYPE_SIZE

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: specialize the settings for size

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: Check for empty output in test_dump_pgstate_history

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20745 from Liuchang0812/jsonify-pool-get

mon: invalid JSON returned when querying pool parameters

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20775 from tspmelo/fix-iscsi-unit-test

mgr/dashboard_v2: add mocked service provider for TcmuIscsiService

Reviewed-by: Volker Theile <vtheile@suse.com>

pybind/ceph_argparse: accept flexible req
True and true both acceptable

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #20778 from p-na/wip-mgr-fix-dashboard-backend-tests

mgr/dashboard: Fix backend tests for newer CherryPy versions

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20782 from sebastian-philipp/dashboard_v2_fix_config_options_endpoint

mgr/dashboard_v2: Fix test_cluster_configuration test

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20786 from dzafman/wip-zafman-log-trim

tools/ceph-objectstore-tool: command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert "make-dist: add OBS-specific release suffix on SUSE"

This reverts commit ca6c92b3ec2f715cf247e9c1009d315d9de9c55a.

The OBS-specific suffix potentially causes trouble when building outside of
OBS. Also, the OBS build process already includes a spec-file munging step
where this can be done.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: Add pg log trim command to ceph-objectstore-tool

Add test script that verifies the command in qa/standalone/osd

Fixes: http://tracker.ceph.com/issues/23242
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #20769 from oritwas/wip-rgw-20448

rgw: fix crash with rgw_run_sync_thread false

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

ceph-volume tests verify that mkfs filestore uses default flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare pass the cephx_secret for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.prepare use keyfile from stdin for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>