mgr/dashboard: skip 'test_create_rbd_in_data_pool' without bluestore

Skip the test case 'test_create_rbd_in_data_pool' only if the cluster
does not support bluestore, otherwise run the test. The Ceph mgr
provides the information if the cluster supports bluestore or not, so
the test will be skipped depending on the current cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add 'osd metadata' command call

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #21019 from majianpeng/fix-vstart

vstart: fix initial start when there is no ceph.conf

Reviewed-by: Mykola Golub <mgolub@suse.com>

vstart.sh: Avoiding ceph-conf error when create a new cluster.

This partly revert commit:f437598cfcabbd66c372bc8.
Before firstly creating a ceph cluster, there is no ceph.conf.
If specify a non-exist ceph.conf, ceph-conf will meet error.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20986 from jecluis/wip-vstart

vstart: fix option (due to quotes) and allow disabling dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20933 from dzafman/wip-22902

src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20989 from ricardoasmarques/wip-npm-start

mgr/dashboard: Improve `npm start` script

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20847 from votdev/improve_auth_interceptor

mgr/dashboard: Improve auth interceptor.

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve auth interceptor.

In case of an error the response must be returned.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20970 from ashishkumsingh/doc-fix-resharding-formatting

doc: Fix dynamic resharding doc formatting

Reviewed-by: Jos Collin <jcollin@redhat.com>

vstart: allow disabling dashboard with option

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20978 from votdev/improve_formatter

mgr/dashboard: Add toBytes() method to FormatterService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20941 from dillaman/wip-dashboard-rbd

mgr/dashboard: additional fixes to block pages

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Implicit use of proxy.conf.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: ng serve should not delete fontend dist dir

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add toBytes() method to FormatterService.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20977 from smithfarm/wip-23424

cmake: add chrono to BOOST_COMPONENTS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20969 from smithfarm/wip-23238

osd: fix s390x build failure

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

vstart: remove single-quotes from option value

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20966 from dillaman/wip-23399

rbd: children list should support snapshot id optional

Reviewed-by: Mykola Golub <mgolub@suse.com>

cmake: add chrono to BOOST_COMPONENTS

According to https://github.com/ceph/ceph/commit/9535165149d16a2a944a81f4888d3ab0a7358027
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.

Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20958 from votdev/remove_useless_code

mgr/dashboard: Remove useless code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rbd: listing children now supports a snapshot id optional

Fixes: http://tracker.ceph.com/issues/23399
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix s390x build failure

46de629c3d0bfae5eba3f6b810808938b5a9af1f introduced a regression: on big-endian
machines, there was no matching function for call to 'eversion_t::encode(const
version_t&, ceph::bufferlist&) const' and 'eversion_t::decode(version_t&,
ceph::buffer::list::iterator&)'

Fixes: http://tracker.ceph.com/issues/23238
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Fixes dynamic-resharding doc formatting

Fixed the formatting errors in dynamicresharding.rst

Signed-off-by: Ashish Singh assingh@redhat.com

Merge pull request #20888 from majianpeng/fix-vstop

stop.sh: add ceph configure file location

Reviewed-by: Mykola Golub <mgolub@suse.com>

pybind/rbd: added 'set_snap_by_id' API method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added '(rbd_)snap_set_by_id' API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internally utilize snap id when setting snapshot

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

stop.sh: specify the ceph.conf path to avoid using wrong ceph.conf.

Also specify the ceph.conf path for ceph-conf command in vstart.sh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20876 from smithfarm/wip-doc-license

legal: remove doc license ambiguity

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #20898 from tspmelo/wip-nodejs-improvement

mgr/dashboard: remove node/npm system installation

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Remove useless code.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20784 from theanalyst/docs/release-reorg

doc: reorganize releases

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20909 from tchaikov/wip-23370

mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20874 from sebastian-philipp/dashboard_v2-source-run-api-tests

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20931 from ashitakasam/fix-typo

doc: s/deamon/daemon/

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: 'daemon' is misspelled in doc/cephfs/health-messages.rst and src/tools/rbd_recover_tool/README

Signed-off-by: ashitakasam <694240887@qq.com>

Merge remote-tracking branch 'gh/mimic-dev2'

Merge pull request #20944 from liewegas/wip-23339

osd: make scrub wait for ec read/modify/writes to apply

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplciatedBackend: do not op_applied on zeroed eversion_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: wait for pushes and last_update_applied before requesting replica scrub

Wait on the primary for the EC backend to finish r/m/w cycles before we
ask the replicas to scrub.  Otherwise we will wait on the primary but the
replicas will scrub immediately and we'll get a mismatched result!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: do not op_applied on zero at_version for dummy rollforward transaction

The dummy rollforward transactions have at_version=0'0; do not call
op_applied on those.

This ensures that the op_applied call

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20948 from liewegas/wip-ecsubread-decode

osd/ECMsgTypes: fix ECSubRead compat decode

Reviewed-by: Myna Vajha <mynaramana@gmail.com>

Merge pull request #20939 from dillaman/wip-cls-rbd-group

cls/rbd: group_image_list incorrectly flagged as RW

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #20903 into master

* refs/pull/20903/head:
PendingReleaseNotes: add note about upgrading MDS

Reviewed-by: Zheng Yan <zyan@redhat.com>

PendingReleaseNotes: add note about upgrading MDS

Fixes: https://tracker.ceph.com/issues/23172
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd/ECMsgTypes: fix ECSubRead compat decode

Populate subchunks based on to_read, not attrs_to_read.

Signed-off-by: Sage Weil <sage@redhat.com>

Partial revert "osd/PG: drop scrub machinery to wait for last_updated_applied"

This reverts commit c489112a1dbcbb63024ba9c8c0abff6ef0c815d8 (well,
part of it).

We want the scrub machinery that will wait on an EC read/modify/write
until it has been queued with the ObjectStore, but we don't need the
parts that used to worry about queued but unapplied writes in FileStore
not being visible.

Fixes: http://tracker.ceph.com/issues/23339
Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: restore op_applied in PGBackend

Call it when we queue the submitted write.  For ReplicatedPG this is
inside submit_transaction(), and for ECBackend it's when we finally
submit the write for disk (possibly delayed by a read/modify/write).

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd/PG: remove last_updated_applied, op_applied()"

This reverts commit c1c495f09754986772c28c9cc7414609572a385c.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #20879 into wip-sage-testing-20180316.201311

* refs/pull/20879/head:
qa/rgw: disable s3tests for lifecycle_expiration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mgr/dashboard: block mirroring page should show Ceph version

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: avoid stale path data in block iSCSI status

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: block iSCSI frontend tweaks

Label rates w/ "/s" suffix and switch throughput back to the
sparkline graph from dashboard v1.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20911 from guzhongyan/ConfM-cleanup

mon/ConfigMonitor: clean up prepare_command()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20914 from ibmibmibm/patch-1

os/memstore: Fix wrong use of lock_guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20930 from badone/wip-Log-delete_type_mismatch

log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Kefu Chai <kchai@redhat.com>

wa/standalong/mon/osd-pool-create: debug and increase delay

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECMsgTypes: print subchunks in ECSubRead operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

This allows us to re-run tests faster. Also updated
HACKING.rst with the latest changes.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20928 from dillaman/wip-dashboard-iscsi

mgr/dashboard: fix 500 error on block device iSCSI status page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20863 from rjfd/wip-qa-fix-ssl-conflict

qa/rgw: renamed ssl task to openssl_keys

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

cls/rbd: group_image_list incorrectly flagged as RW

Fixes: http://tracker.ceph.com/issues/23388
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: fix 500 error on block device iSCSI status page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: remove node/npm system installation

Node and npm are now being installed in a virtualenv, removing the need of
having it installed in the system or as a node dependency.

Now, if you want to use npm, you just need to activate the virtualenv created
on 'build/src/pybind/mgr/dashboard/node-env', and then you can execute the
same commands as you did before.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20925 from rjfd/wip-dashboard-fix-qa

qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/mgr/tasks/dashboard: add fail_on_skip = false

This commit prevents dashboard API tests to be tagged as failed when
some test is decorated with @skip.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20912 from rjfd/wip-dashboard-rename

mgr/dashboard: replace dashboard with dashboard_v2

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: renamed dashboard_v2 to dashboard

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: remove dashboard module

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20916 from tspmelo/fix-mgr-perf-counter

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20892 from guzhongyan/fix-mgrc

mgr: fix MSG_MGR_MAP handling

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19811 from neha-ojha/wip-async-recovery

osd: async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20922 from idryomov/wip-osdmap-encoding-fix

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/mimic-dev2'

# Conflicts:
# qa/standalone/osd/repro_long_log.sh

Merge pull request #20917 from liewegas/wip-no-query-debug

osd: debug dispatch_context cases where queries not sent

Reviewed-by: Josh Durgin <jduring@redhat.com>

Merge pull request #20918 from liewegas/wip-less-many

qa/suites/rados: fewer msgr failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20919 from liewegas/wip-cot

qa/standalone/osd/repro_long_log: no-mon-config for cot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PG: dout cleanup

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add async recovery and backfill to pg dout() prefix

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use clear_location() to clear out missing_loc

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: use get_recovery_read to prevent race during recovery deletes

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle async targets with missing clone in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: send correct stats to async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa: modify TEST_recovery_sizeup() to handle async recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: want set size should not be below min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: add location to missing_loc only during async_recovery

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: clear out old missing_loc in issue_repop

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: handle degraded object in _rollback_to()

Signed-off-by: Neha Ojha <nojha@redhat.com>

ECBackend: update pg_log missing for async_recovery_targets

Signed-off-by: Neha Ojha <nojha@redhat.com>

PG: skip primary in peer_missing in rebuild()

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #20880 into wip-sage-testing-20180315.163409

* refs/pull/20880/head:
"Correct" license from LGPL2 to LGPL2.1

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20910 from ceph/fix_callback_plugins_dir

ceph-volume: fix action plugins path in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20897 from ifed01/wip-ifed-discard-on-mkfs

os/bluestore: do SSD discard on mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20895 from tchaikov/wip-drop-unused-options

common/options: drop unused options

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20445 from aclamk/wip-assert-smaller

common: Make code to invoke assert() smaller.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20887 from liewegas/wip-kill-backlog

osd/osd_types: remove backlog type for pg_log_entry_t

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Currently clients that don't have SERVER_LUMINOUS end up being fed
v6 (i.e. luminous) encoding because they also don't have SERVER_MIMIC.
This was introduced in commit 553048fbf97a ("osd/OSDMap: track newly
removed and purged snaps in each epoch").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>