pybind/mgr: remove type conversions for ceph native options

We now return the correct Py types, just like we do for module options.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: return get_ceph_option result as typed Py object (not string)

We already return module options as the proper Py type; do the same for
native ceph options.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29613 from trociny/wip-41206

librbd: behave more gracefully when data pool removed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #29835 into master

* refs/pull/29835/head:
osd/PeeringState: fix wrong history of merge target

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30097 from jan--f/ceph-backport-cp-indiviudal-commits

ceph-backport.sh: cherry-pick individual commits

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30063 from Rjerk/wip-silence-unused-warning

rgw/services: silence -Wunused-variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30065 from tchaikov/wip-rgw-test-wo-global

test: do not include unnecessary includes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-backport.sh: cherry-pick individual commits

In git 2.23.0 (possibly earlier too) cherry-pick seems to interact with
rev-list and submodules in unexpected ways, where the cherry-pick of a
commit range (<sha1>..<sha1>) fails with "changes would be overwritten"
in a submodule. CP'ing individual commits avoids this.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30060 from c744402859/master

rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30091 from joke-lee/doc-rgw-object-version

rgw: add --object-version in radosgw-admin help info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29957 from smithfarm/wip-ceph-backport-sh-improvements

script/ceph-backport.sh: wholesale refactor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29665 from rishabh-d-dave/cv-remove-redundant-calls

ceph-volume: minor optimizations related to class Volumes's use

rgw: add --object-version in radosgw-admin help info

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge pull request #30086 from uniqueyehu/fix-complie-errors-dpdk

msg/async/dpdk: fix complie errors from fix FTBFS

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/dpdk: fix complie errors from fix FTBFS

Signed-off-by: yehu <yehu5@huawei.com>

Merge pull request #30010 from wjwithagen/wjw-fix-rbd-ggate-postfork

tools/rbd-ggate: close log before running postfork

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29923 from xiexingguo/wip-bump-decode-version

osd/osd_types: bump up some encoding versions

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: yanjun <yan.jun8@zte.com.cn>

Merge pull request #30066 from uniqueyehu/fix-compile-error-dpdk

async/dpdk: fix compile errors from ceph::mutex update

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: do not include unnecessary includes

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".

Signed-off-by: Hongang Chen <c744402859@gmail.com>
Author:    root <root@devel.localdomain>
Date:      Mon Sep 2 11:23:45 2019 +0800

Merge pull request #30034 from tchaikov/wip-install-tox-deps-from-wheelhouse

mgr: install tox deps from wheelhouse

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

async/dpdk: fix compile errors from ceph::mutex update

Signed-off-by: yehu <yehu5@huawei.com>

tools/rbd-ggate: close log before running postfork

Otherwise the assert in Log:open() will trigger since
all data is kept during the fork and also the `is_started` flag

Fixes: https://tracker.ceph.com/issues/41592
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

rgw/services: silence -Wunused-variable warning

Signed-off-by: Lan Liu <liulan@umcloud.com>

script/run_tox.sh: install tox deps from wheelhouse

could be faster

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29583 from rafaelquint/configfield2

mgr/dashboard: Verify fields on Configuration page

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #29919 from dzafman/wip-41522

ceph-objectstore-tool can't remove head with bad snapset

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29941 from dzafman/wip-41517

osd: Fix assert in the case that snapset is missing

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30019 from mxdInspur/fix_20190829

librbd: clean up unused variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29968 from wjwithagen/wjw-fix-dencode-bluestore_bdev_t

bluestore/bdev: initialize size when creating object.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29931 from yaozongyou/refine-decltype

osdc: using decltype(auto) instead of trailing return type

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: clean up unused variable: C_MigrateObject::m_object_size

Fixes: https://tracker.ceph.com/issues/41580
Signed-off-by: mxdInspur <muxiangdong@inspur.com>

Merge pull request #30035 from tchaikov/wip-rgw-maybe-unused

rgw/rgw_user: add [[maybe_unused]] for silencing -Wunused-variable wa…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29927 from votdev/add_cephfs_legend

mgr/dashboard: Display legend for CephFS standbys

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #29934 from votdev/add_missing_i18n

mgr/dashboard: Add missing text translation

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30001 from votdev/issue_41573

mgr/dashboard: A block-manager can not access the pool page

Merge pull request #28885 from Kriechi/fix-bucket-policy-spelling

docs: rgw: fix bucket operation spelling: ListBucketMultipartUploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

script/ceph-backport.sh: leave Assignee blank in backport PR

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Verify fields on Configuration page

Fixes: https://tracker.ceph.com/issues/41193
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

Merge pull request #29794 from votdev/improve_error_msg

mgr/dashboard: Improve RestClient error logging

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #29353 from votdev/issue_40981

mgr/dashboard: CephFS class issues with strings

Merge pull request #29848 from rhcs-dashboard/fix-41358-master

mgr/dashboard: fix SAML input argument handling

Reviewed-by: Lenz Grimmer <LGrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: fix tox missing bits and add lint-fixer (#29802)

mgr/dashboard: fix tox missing bits and add lint-fixer

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

rgw/rgw_user: add [[maybe_unused]] for silencing -Wunused-variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29993 from votdev/fix_pool_e2e

mgr/dashboard: Fix pool deletion e2e

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

mgr/ansible: document requirements in requirements.txt

so install-deps.sh is able to collect them in wheels

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: access_control: add grafana scope read access t… (#29989)

mgr/dashboard: access_control: add grafana scope read access to *-manager roles

Reviewed-by: Lenz Grimmer <LGrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: CephFS class issues with strings

Fixes: https://tracker.ceph.com/issues/40981
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #30022 from Yan-waller/wip-yanj-miscleanup

osd: misc cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: drop useless get_possibly_deleted_pool_pg_num

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

osd: no need to dump empty waiting_for_split

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #29982 from falcon78921/wip-doc-41516

doc: updated ceph monitor config options

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: updated ceph monitor config options

Executed ceph-conf --dump-all on a freshly installed v14.2.2 (nautilus)
cluster. Compared the global defaults to the keys/values specified in
mon-config-ref.rst. Checked options.cc to make sure the obsolete keys
are no longer used.

Fixes: https://tracker.ceph.com/issues/41516
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #29939 from majianpeng/bluestore-print-correctly-info

os/bluestore: print correctly info.

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #30005 from wjwithagen/wjw-fix-vstart-mkdir

vstart.sh: Make sure mkdir succeeds

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix pool deletion e2e

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: A block-manager can not access the pool page

Fixes: https://tracker.ceph.com/issues/41573
Signed-off-by: Volker Theile <vtheile@suse.com>

vstart.sh: Make sure mkdir succeeds

Even in the case the full path was not created, make sure mkdir
does create the full path.

And while there replaced some `test ....` with `[ ... ]`
like in the remainder of the file.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

script/ceph-backport.sh: wholesale refactor

This commit refactors the script to make it more user-friendly and
maintainable. Added features:

* script now determine release/milestone from backport issue instead of
  requiring user to provide it
* script now generates a more verbose PR description
* backport PR title is based on original PR instead of the tracker issue
* improved error handling
* new --debug option that triggers "set -x"
* new --verbose option
* errors and diagnostic messages are now printed to stderr

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph-volume: test number of times LVs list was created

Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #29917 from tchaikov/wip-mon-always-on-modules

mgr/MgrMonitor: print pending.always_on_modules before updating it

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29857 from dzafman/wip-41255

backfill_toofull seen on cluster where the most full OSD is at 1%

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29604 from votdev/python_cleanup

mgr/dashboard: Cleanup Python code

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: access_control: add grafana scope read access to *-manager roles

Fixes: https://tracker.ceph.com/issues/41572
Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Extract E2E wait calls into PageHelper (#29789)

mgr/dashboard: Extract E2E wait calls into PageHelper

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #29881 from tchaikov/wip-test-bufferlist

test/bufferlist.cc: encode/decode int64_t instead of long

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #29916 from badone/wip-tracker-41518-grafana-server

qa/ceph-ansible: Disable dashboard

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Extract E2E wait calls into PageHelper

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge PR #20160 into master

* refs/pull/20160/head:
cephfs: Make mds-mds per message versioned

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29911 into master

* refs/pull/29911/head:
qa: ignore RECENT_CRASH for multimds snapshot testing

Tested-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29966 from idryomov/wip-krbd-zero-after-overlap

qa: krbd_parent_overlap.t: fix read test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29967 from theanalyst/rgw/beast-endpoint-fix

rgw: asio: check the remote endpoint before processing requests

Reviewed-By: Casey Bodley <cbodley@redhat.com>

bluestore/bdev: initialize size when creating object.

The other partss of the struct are initialized by their
ctors. Only for unint_64 there is no ctor.

Otherwise ceph-dencoder tests will fail in comparing
the exported output.
[~/master36] wjw@cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode export /tmp/typ-yFISvjvgj
[~/master36] wjw@cephdev.digiware.nl> hexdump -C !$
hexdump -C /tmp/typ-yFISvjvgj
00000000  62 6c 75 65 73 74 6f 72  65 20 62 6c 6f 63 6b 20 |bluestore block |
00000010  64 65 76 69 63 65 0a 30  30 30 30 30 30 30 30 2d |device.00000000-|
00000020  30 30 30 30 2d 30 30 30  30 2d 30 30 30 30 2d 30 |0000-0000-0000-0|
00000030  30 30 30 30 30 30 30 30  30 30 30 0a 02 01 28 00 |00000000000...(.|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000050  00 00 70 74 03 04 00 00  00 00 00 00 00 00 00 00 |..pt............|
00000060  00 00 00 00 00 00 00 00  00 00 |..........|
0000006a

[~/master36] wjw@cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode decode encode export /tmp/typ-MjWXdCpzJ
[~/master36] wjw@cephdev.digiware.nl> hexdump -C !$
hexdump -C /tmp/typ-MjWXdCpzJ
00000000  62 6c 75 65 73 74 6f 72  65 20 62 6c 6f 63 6b 20 |bluestore block |
00000010  64 65 76 69 63 65 0a 30  30 30 30 30 30 30 30 2d |device.00000000-|
00000020  30 30 30 30 2d 30 30 30  30 2d 30 30 30 30 2d 30 |0000-0000-0000-0|
00000030  30 30 30 30 30 30 30 30  30 30 30 0a 02 01 28 00 |00000000000...(.|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000050  00 00 73 64 00 00 00 00  00 00 00 00 00 00 00 00 |..sd............|
00000060  00 00 00 00 00 00 00 00  00 00 |..........|
0000006a

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

qa: krbd_parent_overlap.t: fix read test

It turns out this test has not been triggering the right code path for
a while now (krbd_parent_overlap.t was added just over a year ago, but
the original test goes back a few years).

To do its job it needs an empty clone with the overlap set to somewhere
in the middle of an object.  The test relied on shrinking for this, but
shrinking no longer fits the bill because a copyup is now performed for
the object with the overlap -- we never go to the parent after that.

Let's size the parent so that it ends in the middle of an object and
then grow an empty clone instead.  Also, while reading from page cache
does seem to trigger a read that straddles the overlap, it's not very
reliable, so supplement it with a direct read.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #29948 from smithfarm/wip-ceph-backport-sh-doc

doc: script/ceph-backport.sh: add "Troubleshooting notes"

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #29950 from smithfarm/wip-bri-deep-scan-of-pr-desc

script/backport-resolve-issue: handle tracker URLs better

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: asio: check the remote endpoint before processing requests

`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #29815 from smanjara/wip-fix-bucket-link

rgw: fix missing tenant prefix in bucket name during bucket link

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

script/backport-resolve-issue: handle tracker URLs better

Before this commit, backport-resolve-issue would blindly grab the first tracker
URL it saw in the GitHub PR description. With this commit, it examines all the
tracker URLs it sees in the PR description and uses the first one that
corresponds to an issue in the Redmine Backport tracker.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: migrate e2e tests to async await (#29758)

mgr/dashboard: migrate e2e tests to async await

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29942 from tchaikov/wip-bump-seastar

seastar: pick up latest changes and cleanups

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

doc: script/ceph-backport.sh: add "Troubleshooting notes"

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: fix protractor.conf.js

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: set sensible `implicitWaitTimeout`

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix issue in dashbboard test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: prepend await before expect statements are used

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix issue E2E image with unclickable element

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E mirroring test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refatoring E2E page helper test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E iscsi test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring Helper class to remove PageHelper instances

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring E2E dashboard test

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: reformat of `protractor.conf.js`

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: refactoring of references of classes to point to classes directly

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix UnhandledPromiseRejectionWarning in RGW users

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E dashboards to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: improve overall E2E stability of navigation

The `navigateTo` method will check if a table exists on the table it has
been used to navigate to and if so, wait for the data of the table to
be loaded.

Fixes: https://tracker.ceph.com/issues/41349
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E user management tests

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E images to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: migrate E2E iscsi to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>