pybind/rbd: added 'set_snap_by_id' API method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added '(rbd_)snap_set_by_id' API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internally utilize snap id when setting snapshot

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20911 from guzhongyan/ConfM-cleanup

mon/ConfigMonitor: clean up prepare_command()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20914 from ibmibmibm/patch-1

os/memstore: Fix wrong use of lock_guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20930 from badone/wip-Log-delete_type_mismatch

log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20928 from dillaman/wip-dashboard-iscsi

mgr/dashboard: fix 500 error on block device iSCSI status page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20863 from rjfd/wip-qa-fix-ssl-conflict

qa/rgw: renamed ssl task to openssl_keys

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: fix 500 error on block device iSCSI status page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20925 from rjfd/wip-dashboard-fix-qa

qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/mgr/tasks/dashboard: add fail_on_skip = false

This commit prevents dashboard API tests to be tagged as failed when
some test is decorated with @skip.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20912 from rjfd/wip-dashboard-rename

mgr/dashboard: replace dashboard with dashboard_v2

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: renamed dashboard_v2 to dashboard

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: remove dashboard module

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20916 from tspmelo/fix-mgr-perf-counter

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20892 from guzhongyan/fix-mgrc

mgr: fix MSG_MGR_MAP handling

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19811 from neha-ojha/wip-async-recovery

osd: async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20922 from idryomov/wip-osdmap-encoding-fix

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/mimic-dev2'

# Conflicts:
# qa/standalone/osd/repro_long_log.sh

Merge pull request #20917 from liewegas/wip-no-query-debug

osd: debug dispatch_context cases where queries not sent

Reviewed-by: Josh Durgin <jduring@redhat.com>

Merge pull request #20918 from liewegas/wip-less-many

qa/suites/rados: fewer msgr failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20919 from liewegas/wip-cot

qa/standalone/osd/repro_long_log: no-mon-config for cot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: dout cleanup

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add async recovery and backfill to pg dout() prefix

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use clear_location() to clear out missing_loc

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use get_recovery_read to prevent race during recovery deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle async targets with missing clone in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: send correct stats to async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa: modify TEST_recovery_sizeup() to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: want set size should not be below min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add location to missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear out old missing_loc in issue_repop

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle degraded object in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: skip primary in peer_missing in rebuild()

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #20880 into wip-sage-testing-20180315.163409

* refs/pull/20880/head:
"Correct" license from LGPL2 to LGPL2.1

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20910 from ceph/fix_callback_plugins_dir

ceph-volume: fix action plugins path in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20897 from ifed01/wip-ifed-discard-on-mkfs

os/bluestore: do SSD discard on mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20895 from tchaikov/wip-drop-unused-options

common/options: drop unused options

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20445 from aclamk/wip-assert-smaller

common: Make code to invoke assert() smaller.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20887 from liewegas/wip-kill-backlog

osd/osd_types: remove backlog type for pg_log_entry_t

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Currently clients that don't have SERVER_LUMINOUS end up being fed
v6 (i.e. luminous) encoding because they also don't have SERVER_MIMIC.
This was introduced in commit 553048fbf97a ("osd/OSDMap: track newly
removed and purged snaps in each epoch").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/standalone/osd/repro_long_log: no-mon-config for cot

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados: fewer msgr failures

500 is a bit much... e.g., enough to hit timeouts forming mon quorum.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: debug dispatch_context cases where queries not sent

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Now that we can obtain perf counters for the mgr, we need to assert that we get
more than 0 counters and that they have the desired structure.

Fixes: https://tracker.ceph.com/issues/23378
Signed-off-by: Tiago Melo <tmelo@suse.com>

memstore: fix wrong use of lock_guard

std::lock_guard need a name to make RAII works properly
Signed-off-by: Shen-Ta Hsieh <ibmibmibm.tw@gmail.com>

ceph-volume: fix action plugins path in tox

because of this commit [1] in ceph-ansible tests in ceph-volume needs to
be modified accordingly.

[1] https://github.com/ceph/ceph-ansible/commit/60d4b75f519c03fca91384f231b071793f582376

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

mon/ConfigMonitor: clean up prepare_command()

no more paxos->get_pending_transaction()

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #20836 from xiexingguo/wip-balancer-05

pybind/mgr/balancer: add sanity check against empty adjusted_map

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20857 from zealoussnow/wip-leozhang

osd/bluestore: delete unused variable in KernelDevice

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20690 from myoungwon/wip-manifest-num-counter

osd: add num_object_manifest

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20652 from shangfufei/wip-common

src/common: update some ms_* options to be more consistent

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #20886 from liewegas/wip-kill-min-max

include: kill MIN and MAX macros

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20321 from jcsp/wip-22918

mgr: initialize PyModuleRegistry sooner

Reviewed-by: Kefu Chai <kchai@redhat.com>

log: Fix AddressSanitizer: new-delete-type-mismatch

 If you directly call the operator new function, you must also directly
 call the operator delete function, and must manually call the
 destructor as well.

Fixes: http://tracker.ceph.com/issues/23324
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

PG: bug fix in is_degraded_or_backfilling_object()

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: reset async_recovery_targets after recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: call choose_acting when there are async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: mark recovery started when fail to get recovery_read_lock

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: set delete flag in needs_recovery_map for deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: update missing_loc for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: do not block writes on async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: modify should_send_op to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

osd: rename ECSubWrite parameter backfill to backfill_or_async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: cleanup and fixes for async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

osd: introduce acting_recovery_backfill shards

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: initial changes required for async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #20901 from neha-ojha/wip-fix-mgr

 qa/tasks/ceph_manager: use set_config on revived osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

 qa/tasks/ceph_manager: use set_config on revived osd

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #20824 from ceph/wip-rm23259

ceph-volume process the abspath of the executable first

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20220 from dzafman/wip-calc-stats3

osd: Improve recovery stat handling by using peer_missing and missing_loc info

Reviewed-by: Sage Weil <sage@redhat.com>

test: osd-backfill-stats.sh parallel osd-recovery-stats.sh check() changes

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Use "(est)" in log message when an osd doesn't have peer_missing

Consolidate check() code and common script code
TEST_recovery_multi() wasn't reliable due to delayed peer_missing

Signed-off-by: David Zafman <dzafman@redhat.com>

test: osd-recovery-stats.sh: New test with different missing objs on multiple OSDs

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix _update_calc_stats() to use peer_missing and missing_by_count

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Changes to handle sharding for EC in missing_by_count

Signed-off-by: David Zafman <dzafman@redhat.com>

osd/PrimaryLogPG: update missing_loc on peer recover

The MissingLoc structure already tracks locations of any missing object
on any acting OSD, and is updated on_local_recover and on_global_recover.
Update it on_peer_recover too so that the loc information (and derivative
missing_by_count info) is correct.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: MissingLoc: track location counts by upset and other

For every object in missing_loc, calculate how many known locations are in
the upset and all others, and call that loc_count_t.  Keep
a map of loc_count_t to the number of missing objects with that
distribution.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>

test: Correction for better degraded/misplaced handling

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Always build peer_missing (including strays) for _update_calc_stats()

Fixes: http://tracker.ceph.com/issues/22837
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix for handling more than 1 missing target

Fix test case to test more than 1 target

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #20751 from s0nea/wip-mgr-dashboard_rbd_add

mgr/dashboard_v2: Add RBD create functionality to the Python backend

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

os/bluestore: trim SSD devices on mkfs

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20878 from ceph/wip-rm23337

ceph-volume document multipath support

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

osd/osd_types: remove backlog type for pg_log_entry_t

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20804 from Devp00l/cephfs-template-fix

mgr/dashboard_v2: Fix cephfs template table usage

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20783 from tspmelo/mgr-perf-counters

mgr/dashboard_v2: add mgr to the list of perf counters

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20728 from jcsp/wip-dashboard-can_run

mgr/dashboard v2: implement can_run method

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

common/options: drop unused options

async_compressor_* have never been used, so drop them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add RBD create functionality

This commit adds a basic RBD create functionality to the REST API. It enables
the POST request of the http://<host>:<port>/api/rbd API endpoint.

The commit adds also related tests.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr: fix MSG_MGR_MAP handling

ceph config show mgr.x doesn't work
root cause is mgr daemon's mgrc has no chance
to process MSG_MGR_MAP in the mgr daemon's
ms_dispatch.

fix is let this message pass through for mgrc

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
Signed-off-by: yupeng chen chenyupeng-it@360.cn

Merge PR #20700 into master

* refs/pull/20700/head:
client: use either dentry_invalidate_cb or remount_cb to invalidate kernel dcache

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #20719 into master

* refs/pull/20719/head:
client: add new ceph_mount_perms_set function

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #20780 into master

* refs/pull/20780/head:
doc/cephfs: mention "ceph fs set-default"
mds: include legacy_client_fscid in FSMap::dump

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

include: kill MIN and MAX macros

Use std::min and std::max instead.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: MIN -> std::min

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #20881 into master

* refs/pull/20881/head:
messages/MClientRequest: fix a typo so we don't output 'setfilelockrule' in logs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>