ceph-volume tests: verify that wipefs tries several times

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Remove button and badge radius (#29001)

mgr/dashboard: Remove button and badge radius

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Fix form validation problems in RGW user form (#29000)

mgr/dashboard: Fix form validation problems in RGW user form

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28267 from flumm/ceph-volume-zap-fix

ceph-volume lvm.zap fix cleanup for db partitions

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Add nolockdep option to e2e-script (#28981)

mgr/dashboard: Add nolockdep option to e2e-script

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #28957 from mgerstner/sfw2

rpm,etc/sysconfig: remove SuSEfirewall2 support

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

packaging: remove SuSEfirewall2 support

removal of SuSEfirewall2 service, since SuSEfirewall2 has been replaced
by firewalld, see [1].

[1]: https://lists.opensuse.org/opensuse-factory/2019-01/msg00490.html

Fixes: http://tracker.ceph.com/issues/40738
Signed-off-by: Matthias Gerstner <matthias.gerstner@suse.de>

Merge pull request #28802 from lixiaoy1/cmake_pmdk

cmake: pmem/pmdk changes to cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28984 from tchaikov/wip-cmake-rebuild-cleanup

cmake: use BUILD_ALWAYS for rebuilding external project

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #29002 into master

* refs/pull/29002/head:
qa/tasks/ceph_deploy: do not rely on ceph-create-keys

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

Merge PR #28643 into master

* refs/pull/28643/head:
doc: removed old EOL Ceph versions from the list
doc: added openSUSE distros to the list and updated the releases
doc: updated the OS recommendations to reflect the Nautilus release

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28944 into master

* refs/pull/28944/head:
qa/workunits/rados/test_cache_pool.sh: s/forward/proxy/
qa/workunits/cephtool/test.sh: cache forward mode is no longer allowed
mon/OSDMonitor: forward -> proxy, readforward -> readproxy on octopus
mon/OSDMonitor: disallow FORWARD and READFORWARD cache modes
osd: do proxy when in forward/readforward cache modes

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Fix form validation problems in RGW user form

ng-bootstrap-form-validation requires that every "form-control" has
an associated "formControlName".

Since these didn't had one, we had to either removed the "form-control" class
or rename it to "cd-form-control" (which will apply the same style without
triggering the validation).

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Remove button and badge radius

Fixes: http://tracker.ceph.com/issues/38953
Signed-off-by: Tiago Melo <tmelo@suse.com>

qa/tasks/ceph_deploy: do not rely on ceph-create-keys

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28426 from mkogan1/wip-rgw-40183

rgw: save an unnecessary copy of RGWEnv

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge PR #28969 into master

* refs/pull/28969/head:
qa/tasks/ceph_manager: make is_{clean,recovered,active_or_down} less racy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #28943 into master

* refs/pull/28943/head:
qa: stop testing on 16.04 xenial

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: do not always build brotli or zstd

these two libraries are relatively stable, and Ceph developers are not
likely to hack them. so no need to rebuild them every time.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake/modules/BuildRocksDB.cmake: use BUILD_ALWAYS

this option was introduced by cmake 3.1, and we require cmake 3.5.1, so
we can use it without using the homebrew solution.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Add nolockdep option to e2e-script

Adding the --nolockdep option to the vstart command
seems to fix the current issue where vstart just
hangs at the dashboard ac-user-create command.
Hopefully we can get the e2e tests to run again
in Jenkins by applying this change.
Note that this might be a workaround only.

Fixes: https://tracker.ceph.com/issues/40688
Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #28766 from liewegas/wip-clean-out-nibblewise

osd/osd_types: drop last_backfill_bitwise member

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28801 from ideepika/wip-update-dump-historic-ops

osd: add duration field to dump_historic_ops method

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>

mgr/dashboard: fix icons variable assignment (#28961)

mgr/dashboard: fix icons variable assignment

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Update bootstrap (#25188)

mgr/dashboard: Update bootstrap

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #28797 from simon-rock/simon_bug_fix

mgr/status: fix 'ceph osd status' ZeroDivisionError

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28687 from hsiang41/mgr/diskprediction_cloud_service_unavailable

mgr/diskprediction_cloud: Service unavailable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28950 from tchaikov/wip-corpus-gen

script,doc: add gen-corpus.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev/corpus.rst: updates to match gen-corpus.sh

* to reference the script to generate corpus
* use qa/workunits/rgw/run-s3tests.sh for producing rgw corpus

Signed-off-by: Kefu Chai <kchai@redhat.com>

script: add gen-corpus.sh

to automate generation of ceph-object-corpus objects. it facilitate the
process of cutting a new release

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28070 from badone/wip-lazy-omap-test

Wip lazy omap test

Reviewed-by: Greg Farnum gfarnum@redhat.com

qa/workunits/rados/test_cache_pool.sh: s/forward/proxy/

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: cache forward mode is no longer allowed

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: forward -> proxy, readforward -> readproxy on octopus

Adjust cache modes on upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: disallow FORWARD and READFORWARD cache modes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28949 from adamemerson/wip-40416

Don't crash on copy when metadata directive not supplied

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/tasks/ceph_manager: make is_{clean,recovered,active_or_down} less racy

Currently these can be thrown off if the cluster is creating or removing
pools at the same time.  Fix by taking a single snapshot of the pg stats
and based our judgement on that.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #28940 into master

* refs/pull/28940/head:
common/config: Remove unused code.

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: Add the support to use system PMDK library

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

cmake: move pmdk build script to Buildpmem.cmake

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

cmake: update HAVE_PMEM to HAVE_BLUESTORE_PMEM

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

mgr/dashboard: Use bootstrap custom-select

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Use ng-bootstrap-form-validation

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Replace awesome-boostrap-checkbox with bootstrap

Fixes: http://tracker.ceph.com/issues/36400
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Switch ng2-toastr for ngx-toastr

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update bootstrap

Fixes: http://tracker.ceph.com/issues/39326
Signed-off-by: Tiago Melo <tmelo@suse.com>

qa: stop testing on 16.04 xenial

Note that the hammer old-client test will now *only* run on centos.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28863 from votdev/fix_various_issues

mgr/dashboard: Fix issues in user form

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

rgw: Don't crash on copy when metadata directive not supplied

Fixes: http://tracker.ceph.com/issues/40416
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mgr/dashboard: fix icons variable assignment

Fixes: https://tracker.ceph.com/issues/40717
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #28897 from ceph/fix-dashboard-hacking_link

mgr/dashboard: Fix link format to HACKING.rst

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28372 from pcuzner/add-mgr-metadata

mgr/prometheus: Add mgr metdata to prometheus exporter module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge PR #28941 into master

* refs/pull/28941/head:
osd: scrub_purged_snaps: fix log message tense
osd/SnapMapper: reset iterators at finish of run()
osd/SnapMapper: drop useless _init()
osd: do not hold osd_lock while queueing stray purged_snaps
osd: more debug output during scrub hang

Reviewed-by: Neha Ojha <nojha@redhat.com>

osd: Modify lazy omap stats to only show one copy

Only show stats from the primary to get more in line with other pg stats
representations.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Add mgr metdata to prometheus exporter module

Add's metadata and status information for the mgr
daemon(s), together with the status of the mgr
modules.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

tests: Add test for lazy omap stat collection

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #28910 from yuvalif/wip-yuval-pubsub-test-fixes

rgw/pubsub: tests enhancements and fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28931 from yuriw/wip-yuriw-40452-master

qa/tests: added ragweed coverage to stress-split* upgrade suites

Merge pull request #28932 from yuriw/wip-yuriw-40452-master_2

qa/tests: added ragweed coverage to stress-split* upgrade suites

common/config: Remove unused code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: scrub_purged_snaps: fix log message tense

Signed-off-by: Sage Weil <sage@redhat.com>

osd/SnapMapper: reset iterators at finish of run()

These iterators implicitly hold locks in FileStore, which can lead to a
deadlock if they are still alive and the caller tries to take new locks.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/SnapMapper: drop useless _init()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #27965 into master

* refs/pull/27965/head:
doc: remove recommendation for kernel.pid_max

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

osd: do proxy when in forward/readforward cache modes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #17614 into master

* refs/pull/17614/head:
doc: cover more cache modes in rados/operations/cache-tiering.rst

Reviewed-by: Sage Weil <sage@redhat.com>

doc: cover more cache modes in rados/operations/cache-tiering.rst

Fixes: http://tracker.ceph.com/issues/14153
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28850 from liewegas/wip-40634

mon/AuthMonitor: fix initial creation of rotating keys

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #28758 into master

* refs/pull/28758/head:
mgr/progress: Created first unit test for progress module
mgr/progress: makes changes to unit test PR
mgr/progress: Created first unit test for progress module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #28842 into master

* refs/pull/28842/head:
test/msgr: make ceph_perf_msgr_client/server work.

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: do not hold osd_lock while queueing stray purged_snaps

Signed-off-by: Sage Weil <sage@redhat.com>

osd: more debug output during scrub hang

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28873 from albIN7/liv3

rgw: fix bugs in listobjectsv1

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #28913 from cyx1231st/wip-seastar-msgr-fix-perfasync

test/crimson: fix interpretability with perf_async_msgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: set python binary for teuthology venv (#28914)

mgr/dashboard: set python binary for teuthology venv

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

rgw: fix bugs in listobjectsv1

Fixes: https://tracker.ceph.com/issues/40663
Signed-off-by: Albin Antony <aantony@redhat.com>

Merge pull request #28920 from tchaikov/wip-cmake-pybind

cmake: avoid rebuilding extensions, and using python-config

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28884 from DimStar77/depchain

ceph.spec.in: Drop systemd BuildRequires in case of building for SUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28843 from tchaikov/wip-armhf-build

cmake, pybind: fix build on armhf

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Fix npm vulnerabilities (#28903)

mgr/dashboard: Fix npm vulnerabilities

Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: set python binary for teuthology venv

* Added env. var. to set python binary when creating teuthology venv.
* venv/bin/activate: run after setting PATH (as it does the last PATH export).

Signed-off-by: alfonsomthd <almartin@redhat.com>

Merge pull request #28888 from joscollin/wip-fix-typo

doc: fix typo

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #28848 into master

* refs/pull/28848/head:
common/blkdev: only pay attention to first 2 bits of smartctl return code
common/blkdev: include stdout and stderr on join error
common/blkdev: fix syntax of error message
common/blkdev: fix smartctl output in error path
common/blkdev: include bad json from smartctl when unparseable
mgr/devicehealth: sort keys when dumping metrics

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge PR #28052 into master

* refs/pull/28052/head:
doc/rgw: document use of 'realm pull' instead of 'period pull'

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

qa/tests: added ragweed coverage to stress-split* upgrade suites

added ceph config set global bluestore_warn_on_legacy_statfs fals
changed user "foo" to "ragweed" in ragweed.py

Fixes: http://tracker.ceph.com/issues/40452
Fices: http://tracker.ceph.com/issues/40467
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/tests: added ragweed coverage to stress-split* upgrade suites

added ceph config set global bluestore_warn_on_legacy_statfs fals
chnaged user "foo" to "ragweed" in ragweed.py

Fixes: http://tracker.ceph.com/issues/40452
Fices: http://tracker.ceph.com/issues/40467
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 5e7e1f382ffbe1515ce6629dda480041260af448)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #28791 from dcasier/bug-40100-2

Return an error, for Bluestore OSD, if WAL or DB are defined in the tags of the OSD but not present on the system

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #28152 from alimaredia/wip-rgw-thread-yield-context-prepare-bucket-index-transaction

rgw: thread option_yield through bucket index transaction prepare

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: change optional_yield to null_yield

In copy_obj_data() in the AtomicObjectProcessor
constructor

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: thread yield context through writing of head obj

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: thread option_yield through bucket index transaction prepare

Signed-off-by: Ali Maredia <amaredia@redhat.com>

pybind: use distutils.sysconfig for compiling flags

this allows maintainer to override the compiling flags when
cross-compiling Ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28823 from cbodley/wip-qa-rgw-swift-server

qa/rgw: more fixes for swift task

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: Silence Alertmanager alerts (#27277)

mgr/dashboard: Silence Alertmanager alerts

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

test/crimson: default bind to 127.0.0.1 for perf

Async-messenger's accepting in v2 will fail if it binds to a blank ip.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: add v1-crc-enabled for msgr perf

* fix a bug that perf_crimson_msgr and perf_async_msgr cannot connect
  to each other in v1 due to CRC checks;
* add an option to enable v1 CRC checks, because they are enabled in
  real world, and we might want to see the performance impact;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: track msg with tid in perf_async_msgr

fix perf_async_msgr by setting tid for perf_crimson_msgr to track
messages.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge PR #28901 into master

* refs/pull/28901/head:
qa/standalone/scrub/osd-scrub-repair: fix 'scrub ok' grep
osd/osd_types: remove 'snap_context' from SnapSet::dump()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake/modules/Distutils.cmake: add setup as a dependency

so the extension will be rebuilt if setup.py is updated

Signed-off-by: Kefu Chai <kchai@redhat.com>