vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Setting it via `ceph config`, when the cluster is already built and
started is too late, because it is used when building the initial osd
map.

Fixes: https://tracker.ceph.com/issues/23430
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20986 from jecluis/wip-vstart

vstart: fix option (due to quotes) and allow disabling dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20933 from dzafman/wip-22902

src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20989 from ricardoasmarques/wip-npm-start

mgr/dashboard: Improve `npm start` script

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20847 from votdev/improve_auth_interceptor

mgr/dashboard: Improve auth interceptor.

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve auth interceptor.

In case of an error the response must be returned.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20970 from ashishkumsingh/doc-fix-resharding-formatting

doc: Fix dynamic resharding doc formatting

Reviewed-by: Jos Collin <jcollin@redhat.com>

vstart: allow disabling dashboard with option

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20978 from votdev/improve_formatter

mgr/dashboard: Add toBytes() method to FormatterService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20941 from dillaman/wip-dashboard-rbd

mgr/dashboard: additional fixes to block pages

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Implicit use of proxy.conf.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: ng serve should not delete fontend dist dir

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add toBytes() method to FormatterService.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20977 from smithfarm/wip-23424

cmake: add chrono to BOOST_COMPONENTS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20969 from smithfarm/wip-23238

osd: fix s390x build failure

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

vstart: remove single-quotes from option value

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20966 from dillaman/wip-23399

rbd: children list should support snapshot id optional

Reviewed-by: Mykola Golub <mgolub@suse.com>

cmake: add chrono to BOOST_COMPONENTS

According to https://github.com/ceph/ceph/commit/9535165149d16a2a944a81f4888d3ab0a7358027
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.

Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20958 from votdev/remove_useless_code

mgr/dashboard: Remove useless code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rbd: listing children now supports a snapshot id optional

Fixes: http://tracker.ceph.com/issues/23399
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix s390x build failure

46de629c3d0bfae5eba3f6b810808938b5a9af1f introduced a regression: on big-endian
machines, there was no matching function for call to 'eversion_t::encode(const
version_t&, ceph::bufferlist&) const' and 'eversion_t::decode(version_t&,
ceph::buffer::list::iterator&)'

Fixes: http://tracker.ceph.com/issues/23238
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Fixes dynamic-resharding doc formatting

Fixed the formatting errors in dynamicresharding.rst

Signed-off-by: Ashish Singh assingh@redhat.com

Merge pull request #20888 from majianpeng/fix-vstop

stop.sh: add ceph configure file location

Reviewed-by: Mykola Golub <mgolub@suse.com>

pybind/rbd: added 'set_snap_by_id' API method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added '(rbd_)snap_set_by_id' API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internally utilize snap id when setting snapshot

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

stop.sh: specify the ceph.conf path to avoid using wrong ceph.conf.

Also specify the ceph.conf path for ceph-conf command in vstart.sh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20876 from smithfarm/wip-doc-license

legal: remove doc license ambiguity

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #20898 from tspmelo/wip-nodejs-improvement

mgr/dashboard: remove node/npm system installation

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Remove useless code.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20784 from theanalyst/docs/release-reorg

doc: reorganize releases

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20909 from tchaikov/wip-23370

mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20874 from sebastian-philipp/dashboard_v2-source-run-api-tests

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20931 from ashitakasam/fix-typo

doc: s/deamon/daemon/

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: 'daemon' is misspelled in doc/cephfs/health-messages.rst and src/tools/rbd_recover_tool/README

Signed-off-by: ashitakasam <694240887@qq.com>

Merge remote-tracking branch 'gh/mimic-dev2'

Merge pull request #20944 from liewegas/wip-23339

osd: make scrub wait for ec read/modify/writes to apply

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplciatedBackend: do not op_applied on zeroed eversion_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: wait for pushes and last_update_applied before requesting replica scrub

Wait on the primary for the EC backend to finish r/m/w cycles before we
ask the replicas to scrub.  Otherwise we will wait on the primary but the
replicas will scrub immediately and we'll get a mismatched result!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: do not op_applied on zero at_version for dummy rollforward transaction

The dummy rollforward transactions have at_version=0'0; do not call
op_applied on those.

This ensures that the op_applied call

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20948 from liewegas/wip-ecsubread-decode

osd/ECMsgTypes: fix ECSubRead compat decode

Reviewed-by: Myna Vajha <mynaramana@gmail.com>

Merge pull request #20939 from dillaman/wip-cls-rbd-group

cls/rbd: group_image_list incorrectly flagged as RW

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #20903 into master

* refs/pull/20903/head:
PendingReleaseNotes: add note about upgrading MDS

Reviewed-by: Zheng Yan <zyan@redhat.com>

PendingReleaseNotes: add note about upgrading MDS

Fixes: https://tracker.ceph.com/issues/23172
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd/ECMsgTypes: fix ECSubRead compat decode

Populate subchunks based on to_read, not attrs_to_read.

Signed-off-by: Sage Weil <sage@redhat.com>

Partial revert "osd/PG: drop scrub machinery to wait for last_updated_applied"

This reverts commit c489112a1dbcbb63024ba9c8c0abff6ef0c815d8 (well,
part of it).

We want the scrub machinery that will wait on an EC read/modify/write
until it has been queued with the ObjectStore, but we don't need the
parts that used to worry about queued but unapplied writes in FileStore
not being visible.

Fixes: http://tracker.ceph.com/issues/23339
Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: restore op_applied in PGBackend

Call it when we queue the submitted write.  For ReplicatedPG this is
inside submit_transaction(), and for ECBackend it's when we finally
submit the write for disk (possibly delayed by a read/modify/write).

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd/PG: remove last_updated_applied, op_applied()"

This reverts commit c1c495f09754986772c28c9cc7414609572a385c.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #20879 into wip-sage-testing-20180316.201311

* refs/pull/20879/head:
qa/rgw: disable s3tests for lifecycle_expiration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mgr/dashboard: block mirroring page should show Ceph version

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: avoid stale path data in block iSCSI status

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: block iSCSI frontend tweaks

Label rates w/ "/s" suffix and switch throughput back to the
sparkline graph from dashboard v1.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20911 from guzhongyan/ConfM-cleanup

mon/ConfigMonitor: clean up prepare_command()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20914 from ibmibmibm/patch-1

os/memstore: Fix wrong use of lock_guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20930 from badone/wip-Log-delete_type_mismatch

log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Kefu Chai <kchai@redhat.com>

wa/standalong/mon/osd-pool-create: debug and increase delay

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECMsgTypes: print subchunks in ECSubRead operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

This allows us to re-run tests faster. Also updated
HACKING.rst with the latest changes.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20928 from dillaman/wip-dashboard-iscsi

mgr/dashboard: fix 500 error on block device iSCSI status page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20863 from rjfd/wip-qa-fix-ssl-conflict

qa/rgw: renamed ssl task to openssl_keys

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

cls/rbd: group_image_list incorrectly flagged as RW

Fixes: http://tracker.ceph.com/issues/23388
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: fix 500 error on block device iSCSI status page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: remove node/npm system installation

Node and npm are now being installed in a virtualenv, removing the need of
having it installed in the system or as a node dependency.

Now, if you want to use npm, you just need to activate the virtualenv created
on 'build/src/pybind/mgr/dashboard/node-env', and then you can execute the
same commands as you did before.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20925 from rjfd/wip-dashboard-fix-qa

qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/mgr/tasks/dashboard: add fail_on_skip = false

This commit prevents dashboard API tests to be tagged as failed when
some test is decorated with @skip.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20912 from rjfd/wip-dashboard-rename

mgr/dashboard: replace dashboard with dashboard_v2

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: renamed dashboard_v2 to dashboard

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: remove dashboard module

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20916 from tspmelo/fix-mgr-perf-counter

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20892 from guzhongyan/fix-mgrc

mgr: fix MSG_MGR_MAP handling

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19811 from neha-ojha/wip-async-recovery

osd: async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20922 from idryomov/wip-osdmap-encoding-fix

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/mimic-dev2'

# Conflicts:
# qa/standalone/osd/repro_long_log.sh

Merge pull request #20917 from liewegas/wip-no-query-debug

osd: debug dispatch_context cases where queries not sent

Reviewed-by: Josh Durgin <jduring@redhat.com>

Merge pull request #20918 from liewegas/wip-less-many

qa/suites/rados: fewer msgr failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20919 from liewegas/wip-cot

qa/standalone/osd/repro_long_log: no-mon-config for cot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: dout cleanup

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add async recovery and backfill to pg dout() prefix

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use clear_location() to clear out missing_loc

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use get_recovery_read to prevent race during recovery deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle async targets with missing clone in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: send correct stats to async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa: modify TEST_recovery_sizeup() to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: want set size should not be below min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add location to missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear out old missing_loc in issue_repop

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle degraded object in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: skip primary in peer_missing in rebuild()

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #20880 into wip-sage-testing-20180315.163409

* refs/pull/20880/head:
"Correct" license from LGPL2 to LGPL2.1

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20910 from ceph/fix_callback_plugins_dir

ceph-volume: fix action plugins path in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20897 from ifed01/wip-ifed-discard-on-mkfs

os/bluestore: do SSD discard on mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20895 from tchaikov/wip-drop-unused-options

common/options: drop unused options

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20445 from aclamk/wip-assert-smaller

common: Make code to invoke assert() smaller.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20887 from liewegas/wip-kill-backlog

osd/osd_types: remove backlog type for pg_log_entry_t

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Currently clients that don't have SERVER_LUMINOUS end up being fed
v6 (i.e. luminous) encoding because they also don't have SERVER_MIMIC.
This was introduced in commit 553048fbf97a ("osd/OSDMap: track newly
removed and purged snaps in each epoch").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/standalone/osd/repro_long_log: no-mon-config for cot

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados: fewer msgr failures

500 is a bit much... e.g., enough to hit timeouts forming mon quorum.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: debug dispatch_context cases where queries not sent

Signed-off-by: Sage Weil <sage@redhat.com>