os/bluestore: fix some code formatting

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #21002 from smithfarm/wip-23437

build/ops: setup-virtualenv.sh: do not hardcode python binary

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21013 from dzafman/wip-remove-bc

test: Replace bc command with printf command

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21008 from trociny/wip-23430

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #20981 from cbodley/wip-qa-fix-ssl

qa: fix overrides for openssl_keys task

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #20841 from oritwas/wip-rgw-metadata

rgw: trim all spaces inside a metadata value

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20842 from mkogan1/wip-rgw-23147

rgw: enable 'qlen' & 'qactive' performance counters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20846 from theanalyst/rgw/fix-empty-stats

rgw: radosgw-admin abort early for user stats for empty uids

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21021 from sebastian-philipp/dashboard-auth-qa-fix

qa/tasks/mgr/dashboard: Fix login expires too soon

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20972 from ricardoasmarques/wip-dimless-binary-directive

mgr/dashboard: Add DimlessBinaryDirective

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20971 from ricardoasmarques/wip-helper-component

mgr/dashboard: Add helper component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add helper component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

tools: setup-virtualenv.sh: improve error message

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: add usage function and --help option

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/mgr/dashboard: Fix login expires too soon

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

tools: setup-virtualenv.sh: check for mandatory argument

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: declare python version when calling setup-virtualenv.sh

This just makes the current Python 2.7 dependency explicit. Making
these statements optionally use Python 3 is left for later.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: do not hardcode python binary

. . . instead, take a command-line option that defaults to
"python2.7"

Fixes: http://tracker.ceph.com/issues/23437
Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Add DimlessBinaryDirective

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21019 from majianpeng/fix-vstart

vstart: fix initial start when there is no ceph.conf

Reviewed-by: Mykola Golub <mgolub@suse.com>

vstart.sh: Avoiding ceph-conf error when create a new cluster.

This partly revert commit:f437598cfcabbd66c372bc8.
Before firstly creating a ceph cluster, there is no ceph.conf.
If specify a non-exist ceph.conf, ceph-conf will meet error.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test: Replace bc command with printf command

Signed-off-by: David Zafman <dzafman@redhat.com>

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Setting it via `ceph config`, when the cluster is already built and
started is too late, because it is used when building the initial osd
map.

Fixes: https://tracker.ceph.com/issues/23430
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20986 from jecluis/wip-vstart

vstart: fix option (due to quotes) and allow disabling dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20933 from dzafman/wip-22902

src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: fix overrides for openssl_keys task

teuthology looks for overrides by the class name, so overrides weren't
working after the task was renamed

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20989 from ricardoasmarques/wip-npm-start

mgr/dashboard: Improve `npm start` script

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20847 from votdev/improve_auth_interceptor

mgr/dashboard: Improve auth interceptor.

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve auth interceptor.

In case of an error the response must be returned.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20970 from ashishkumsingh/doc-fix-resharding-formatting

doc: Fix dynamic resharding doc formatting

Reviewed-by: Jos Collin <jcollin@redhat.com>

vstart: allow disabling dashboard with option

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mgr/dashboard: Add support for EiB-YiB on dimless.pipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add support for EiB-YiB on dimless-binary.pipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #20978 from votdev/improve_formatter

mgr/dashboard: Add toBytes() method to FormatterService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20941 from dillaman/wip-dashboard-rbd

mgr/dashboard: additional fixes to block pages

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Implicit use of proxy.conf.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: ng serve should not delete fontend dist dir

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add toBytes() method to FormatterService.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20977 from smithfarm/wip-23424

cmake: add chrono to BOOST_COMPONENTS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20969 from smithfarm/wip-23238

osd: fix s390x build failure

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

vstart: remove single-quotes from option value

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20966 from dillaman/wip-23399

rbd: children list should support snapshot id optional

Reviewed-by: Mykola Golub <mgolub@suse.com>

cmake: add chrono to BOOST_COMPONENTS

According to https://github.com/ceph/ceph/commit/9535165149d16a2a944a81f4888d3ab0a7358027
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.

Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20958 from votdev/remove_useless_code

mgr/dashboard: Remove useless code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rbd: listing children now supports a snapshot id optional

Fixes: http://tracker.ceph.com/issues/23399
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix s390x build failure

46de629c3d0bfae5eba3f6b810808938b5a9af1f introduced a regression: on big-endian
machines, there was no matching function for call to 'eversion_t::encode(const
version_t&, ceph::bufferlist&) const' and 'eversion_t::decode(version_t&,
ceph::buffer::list::iterator&)'

Fixes: http://tracker.ceph.com/issues/23238
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Fixes dynamic-resharding doc formatting

Fixed the formatting errors in dynamicresharding.rst

Signed-off-by: Ashish Singh assingh@redhat.com

Merge pull request #20888 from majianpeng/fix-vstop

stop.sh: add ceph configure file location

Reviewed-by: Mykola Golub <mgolub@suse.com>

pybind/rbd: added 'set_snap_by_id' API method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added '(rbd_)snap_set_by_id' API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internally utilize snap id when setting snapshot

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

stop.sh: specify the ceph.conf path to avoid using wrong ceph.conf.

Also specify the ceph.conf path for ceph-conf command in vstart.sh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20876 from smithfarm/wip-doc-license

legal: remove doc license ambiguity

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #20898 from tspmelo/wip-nodejs-improvement

mgr/dashboard: remove node/npm system installation

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Remove useless code.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20784 from theanalyst/docs/release-reorg

doc: reorganize releases

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20909 from tchaikov/wip-23370

mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20874 from sebastian-philipp/dashboard_v2-source-run-api-tests

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20931 from ashitakasam/fix-typo

doc: s/deamon/daemon/

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: 'daemon' is misspelled in doc/cephfs/health-messages.rst and src/tools/rbd_recover_tool/README

Signed-off-by: ashitakasam <694240887@qq.com>

Merge remote-tracking branch 'gh/mimic-dev2'

Merge pull request #20944 from liewegas/wip-23339

osd: make scrub wait for ec read/modify/writes to apply

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplciatedBackend: do not op_applied on zeroed eversion_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: wait for pushes and last_update_applied before requesting replica scrub

Wait on the primary for the EC backend to finish r/m/w cycles before we
ask the replicas to scrub.  Otherwise we will wait on the primary but the
replicas will scrub immediately and we'll get a mismatched result!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: do not op_applied on zero at_version for dummy rollforward transaction

The dummy rollforward transactions have at_version=0'0; do not call
op_applied on those.

This ensures that the op_applied call

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20948 from liewegas/wip-ecsubread-decode

osd/ECMsgTypes: fix ECSubRead compat decode

Reviewed-by: Myna Vajha <mynaramana@gmail.com>

Merge pull request #20939 from dillaman/wip-cls-rbd-group

cls/rbd: group_image_list incorrectly flagged as RW

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #20903 into master

* refs/pull/20903/head:
PendingReleaseNotes: add note about upgrading MDS

Reviewed-by: Zheng Yan <zyan@redhat.com>

PendingReleaseNotes: add note about upgrading MDS

Fixes: https://tracker.ceph.com/issues/23172
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd/ECMsgTypes: fix ECSubRead compat decode

Populate subchunks based on to_read, not attrs_to_read.

Signed-off-by: Sage Weil <sage@redhat.com>

Partial revert "osd/PG: drop scrub machinery to wait for last_updated_applied"

This reverts commit c489112a1dbcbb63024ba9c8c0abff6ef0c815d8 (well,
part of it).

We want the scrub machinery that will wait on an EC read/modify/write
until it has been queued with the ObjectStore, but we don't need the
parts that used to worry about queued but unapplied writes in FileStore
not being visible.

Fixes: http://tracker.ceph.com/issues/23339
Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: restore op_applied in PGBackend

Call it when we queue the submitted write.  For ReplicatedPG this is
inside submit_transaction(), and for ECBackend it's when we finally
submit the write for disk (possibly delayed by a read/modify/write).

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd/PG: remove last_updated_applied, op_applied()"

This reverts commit c1c495f09754986772c28c9cc7414609572a385c.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #20879 into wip-sage-testing-20180316.201311

* refs/pull/20879/head:
qa/rgw: disable s3tests for lifecycle_expiration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mgr/dashboard: block mirroring page should show Ceph version

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: avoid stale path data in block iSCSI status

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: block iSCSI frontend tweaks

Label rates w/ "/s" suffix and switch throughput back to the
sparkline graph from dashboard v1.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20911 from guzhongyan/ConfM-cleanup

mon/ConfigMonitor: clean up prepare_command()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20914 from ibmibmibm/patch-1

os/memstore: Fix wrong use of lock_guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20930 from badone/wip-Log-delete_type_mismatch

log: Fix AddressSanitizer: new-delete-type-mismatch

Reviewed-by: Kefu Chai <kchai@redhat.com>

wa/standalong/mon/osd-pool-create: debug and increase delay

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECMsgTypes: print subchunks in ECSubRead operator<<

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

This allows us to re-run tests faster. Also updated
HACKING.rst with the latest changes.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #20928 from dillaman/wip-dashboard-iscsi

mgr/dashboard: fix 500 error on block device iSCSI status page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20863 from rjfd/wip-qa-fix-ssl-conflict

qa/rgw: renamed ssl task to openssl_keys

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

cls/rbd: group_image_list incorrectly flagged as RW

Fixes: http://tracker.ceph.com/issues/23388
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: fix 500 error on block device iSCSI status page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: remove node/npm system installation

Node and npm are now being installed in a virtualenv, removing the need of
having it installed in the system or as a node dependency.

Now, if you want to use npm, you just need to activate the virtualenv created
on 'build/src/pybind/mgr/dashboard/node-env', and then you can execute the
same commands as you did before.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20925 from rjfd/wip-dashboard-fix-qa

qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/mgr/tasks/dashboard: add fail_on_skip = false

This commit prevents dashboard API tests to be tagged as failed when
some test is decorated with @skip.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20912 from rjfd/wip-dashboard-rename

mgr/dashboard: replace dashboard with dashboard_v2

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: renamed dashboard_v2 to dashboard

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: remove dashboard module

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20916 from tspmelo/fix-mgr-perf-counter

mgr/dashboard_v2: fix test_perf_counters_mgr_get

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20892 from guzhongyan/fix-mgrc

mgr: fix MSG_MGR_MAP handling

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19811 from neha-ojha/wip-async-recovery

osd: async recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20922 from idryomov/wip-osdmap-encoding-fix

osd/OSDMap: fix HAVE_FEATURE logic in encode()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/mimic-dev2'

# Conflicts:
# qa/standalone/osd/repro_long_log.sh

Merge pull request #20917 from liewegas/wip-no-query-debug

osd: debug dispatch_context cases where queries not sent

Reviewed-by: Josh Durgin <jduring@redhat.com>

Merge pull request #20918 from liewegas/wip-less-many

qa/suites/rados: fewer msgr failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>