intarith: kill P2* macros

These macros are originally introduced in https://github.com/ceph/ceph/pull/10128
and get replaced by https://github.com/ceph/ceph/pull/19913.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20931 from ashitakasam/fix-typo

doc: s/deamon/daemon/

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: 'daemon' is misspelled in doc/cephfs/health-messages.rst and src/tools/rbd_recover_tool/README

Signed-off-by: ashitakasam <694240887@qq.com>

Merge remote-tracking branch 'gh/mimic-dev2'

Merge pull request #20944 from liewegas/wip-23339

osd: make scrub wait for ec read/modify/writes to apply

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplciatedBackend: do not op_applied on zeroed eversion_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: wait for pushes and last_update_applied before requesting replica scrub

Wait on the primary for the EC backend to finish r/m/w cycles before we
ask the replicas to scrub.  Otherwise we will wait on the primary but the
replicas will scrub immediately and we'll get a mismatched result!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: do not op_applied on zero at_version for dummy rollforward transaction

The dummy rollforward transactions have at_version=0'0; do not call
op_applied on those.

This ensures that the op_applied call

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20948 from liewegas/wip-ecsubread-decode

osd/ECMsgTypes: fix ECSubRead compat decode

Reviewed-by: Myna Vajha <mynaramana@gmail.com>

Merge pull request #20939 from dillaman/wip-cls-rbd-group

cls/rbd: group_image_list incorrectly flagged as RW

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #20903 into master

* refs/pull/20903/head:
PendingReleaseNotes: add note about upgrading MDS

Reviewed-by: Zheng Yan <zyan@redhat.com>

PendingReleaseNotes: add note about upgrading MDS

Fixes: https://tracker.ceph.com/issues/23172
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd/ECMsgTypes: fix ECSubRead compat decode

Populate subchunks based on to_read, not attrs_to_read.

Signed-off-by: Sage Weil <sage@redhat.com>

Partial revert "osd/PG: drop scrub machinery to wait for last_updated_applied"

This reverts commit c489112a1dbcbb63024ba9c8c0abff6ef0c815d8 (well,
part of it).

We want the scrub machinery that will wait on an EC read/modify/write
until it has been queued with the ObjectStore, but we don't need the
parts that used to worry about queued but unapplied writes in FileStore
not being visible.

Fixes: http://tracker.ceph.com/issues/23339
Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: restore op_applied in PGBackend

Call it when we queue the submitted write.  For ReplicatedPG this is
inside submit_transaction(), and for ECBackend it's when we finally
submit the write for disk (possibly delayed by a read/modify/write).

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd/PG: remove last_updated_applied, op_applied()"

This reverts commit c1c495f09754986772c28c9cc7414609572a385c.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #20879 into wip-sage-testing-20180316.201311

* refs/pull/20879/head:
qa/rgw: disable s3tests for lifecycle_expiration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #20911 from guzhongyan/ConfM-cleanup

mon/ConfigMonitor: clean up prepare_command()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20914 from ibmibmibm/patch-1

os/memstore: Fix wrong use of lock_guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20930 from badone/wip-Log-delete_type_mismatch

log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Kefu Chai <kchai@redhat.com>

wa/standalong/mon/osd-pool-create: debug and increase delay

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECMsgTypes: print subchunks in ECSubRead operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20928 from dillaman/wip-dashboard-iscsi

mgr/dashboard: fix 500 error on block device iSCSI status page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20863 from rjfd/wip-qa-fix-ssl-conflict

qa/rgw: renamed ssl task to openssl_keys

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

cls/rbd: group_image_list incorrectly flagged as RW

Fixes: http://tracker.ceph.com/issues/23388
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: fix 500 error on block device iSCSI status page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20925 from rjfd/wip-dashboard-fix-qa

qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/mgr/tasks/dashboard: add fail_on_skip = false

This commit prevents dashboard API tests to be tagged as failed when
some test is decorated with @skip.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20912 from rjfd/wip-dashboard-rename

mgr/dashboard: replace dashboard with dashboard_v2

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: renamed dashboard_v2 to dashboard

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: remove dashboard module

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20916 from tspmelo/fix-mgr-perf-counter

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20892 from guzhongyan/fix-mgrc

mgr: fix MSG_MGR_MAP handling

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19811 from neha-ojha/wip-async-recovery

osd: async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20922 from idryomov/wip-osdmap-encoding-fix

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/mimic-dev2'

# Conflicts:
# qa/standalone/osd/repro_long_log.sh

Merge pull request #20917 from liewegas/wip-no-query-debug

osd: debug dispatch_context cases where queries not sent

Reviewed-by: Josh Durgin <jduring@redhat.com>

Merge pull request #20918 from liewegas/wip-less-many

qa/suites/rados: fewer msgr failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20919 from liewegas/wip-cot

qa/standalone/osd/repro_long_log: no-mon-config for cot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: dout cleanup

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add async recovery and backfill to pg dout() prefix

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use clear_location() to clear out missing_loc

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use get_recovery_read to prevent race during recovery deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle async targets with missing clone in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: send correct stats to async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa: modify TEST_recovery_sizeup() to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: want set size should not be below min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add location to missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear out old missing_loc in issue_repop

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle degraded object in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: skip primary in peer_missing in rebuild()

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #20880 into wip-sage-testing-20180315.163409

* refs/pull/20880/head:
"Correct" license from LGPL2 to LGPL2.1

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20910 from ceph/fix_callback_plugins_dir

ceph-volume: fix action plugins path in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20897 from ifed01/wip-ifed-discard-on-mkfs

os/bluestore: do SSD discard on mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20895 from tchaikov/wip-drop-unused-options

common/options: drop unused options

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20445 from aclamk/wip-assert-smaller

common: Make code to invoke assert() smaller.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20887 from liewegas/wip-kill-backlog

osd/osd_types: remove backlog type for pg_log_entry_t

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Currently clients that don't have SERVER_LUMINOUS end up being fed
v6 (i.e. luminous) encoding because they also don't have SERVER_MIMIC.
This was introduced in commit 553048fbf97a ("osd/OSDMap: track newly
removed and purged snaps in each epoch").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/standalone/osd/repro_long_log: no-mon-config for cot

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados: fewer msgr failures

500 is a bit much... e.g., enough to hit timeouts forming mon quorum.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: debug dispatch_context cases where queries not sent

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Now that we can obtain perf counters for the mgr, we need to assert that we get
more than 0 counters and that they have the desired structure.

Fixes: https://tracker.ceph.com/issues/23378
Signed-off-by: Tiago Melo <tmelo@suse.com>

memstore: fix wrong use of lock_guard

std::lock_guard need a name to make RAII works properly
Signed-off-by: Shen-Ta Hsieh <ibmibmibm.tw@gmail.com>

ceph-volume: fix action plugins path in tox

because of this commit [1] in ceph-ansible tests in ceph-volume needs to
be modified accordingly.

[1] https://github.com/ceph/ceph-ansible/commit/60d4b75f519c03fca91384f231b071793f582376

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

mon/ConfigMonitor: clean up prepare_command()

no more paxos->get_pending_transaction()

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #20836 from xiexingguo/wip-balancer-05

pybind/mgr/balancer: add sanity check against empty adjusted_map

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20857 from zealoussnow/wip-leozhang

osd/bluestore: delete unused variable in KernelDevice

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20690 from myoungwon/wip-manifest-num-counter

osd: add num_object_manifest

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20652 from shangfufei/wip-common

src/common: update some ms_* options to be more consistent

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #20886 from liewegas/wip-kill-min-max

include: kill MIN and MAX macros

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20321 from jcsp/wip-22918

mgr: initialize PyModuleRegistry sooner

Reviewed-by: Kefu Chai <kchai@redhat.com>

log: Fix AddressSanitizer: new-delete-type-mismatch

 If you directly call the operator new function, you must also directly
 call the operator delete function, and must manually call the
 destructor as well.

Fixes: http://tracker.ceph.com/issues/23324
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

PG: bug fix in is_degraded_or_backfilling_object()

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: reset async_recovery_targets after recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: call choose_acting when there are async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: mark recovery started when fail to get recovery_read_lock

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: set delete flag in needs_recovery_map for deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: update missing_loc for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: do not block writes on async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: modify should_send_op to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

osd: rename ECSubWrite parameter backfill to backfill_or_async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: cleanup and fixes for async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

osd: introduce acting_recovery_backfill shards

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: initial changes required for async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #20901 from neha-ojha/wip-fix-mgr

 qa/tasks/ceph_manager: use set_config on revived osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

 qa/tasks/ceph_manager: use set_config on revived osd

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #20824 from ceph/wip-rm23259

ceph-volume process the abspath of the executable first

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20220 from dzafman/wip-calc-stats3

osd: Improve recovery stat handling by using peer_missing and missing_loc info

Reviewed-by: Sage Weil <sage@redhat.com>

test: osd-backfill-stats.sh parallel osd-recovery-stats.sh check() changes

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Use "(est)" in log message when an osd doesn't have peer_missing

Consolidate check() code and common script code
TEST_recovery_multi() wasn't reliable due to delayed peer_missing

Signed-off-by: David Zafman <dzafman@redhat.com>

test: osd-recovery-stats.sh: New test with different missing objs on multiple OSDs

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix _update_calc_stats() to use peer_missing and missing_by_count

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Changes to handle sharding for EC in missing_by_count

Signed-off-by: David Zafman <dzafman@redhat.com>

osd/PrimaryLogPG: update missing_loc on peer recover

The MissingLoc structure already tracks locations of any missing object
on any acting OSD, and is updated on_local_recover and on_global_recover.
Update it on_peer_recover too so that the loc information (and derivative
missing_by_count info) is correct.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: MissingLoc: track location counts by upset and other

For every object in missing_loc, calculate how many known locations are in
the upset and all others, and call that loc_count_t.  Keep
a map of loc_count_t to the number of missing objects with that
distribution.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>

test: Correction for better degraded/misplaced handling

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Always build peer_missing (including strays) for _update_calc_stats()

Fixes: http://tracker.ceph.com/issues/22837
Signed-off-by: David Zafman <dzafman@redhat.com>