common: fix a typo.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

rbd-mirror: fix two typos.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

Merge pull request #25106 from tchaikov/wip-qa-envlibrados

qa: fix upgrade tests and test_envlibrados_for_rocksdb.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: patch rocksdb the right way

quote from patch(1)

>       but usually just
>
>       patch -pnum <patchfile

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24979 from neha-ojha/wip-upgrade-bug

doc/releases/mimic.rst: make note of 13.2.2 upgrade bug

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #25072 from tchaikov/wip-silence-sign-compare

os/tests: silence -Wsign-compare warning

Reviewed-by: Jos Collin <jcollin@redhat.com>

qa: use FOUND_VAR to be backward compatible with cmake 2.8.12

before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html

while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs

in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.

after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.

also, the message() commands for debugging purpose are removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites: add librados2 to "extra_packages" for upgrade tests

we use the playbook of "testnodes.yml" defined by ceph-cm-ansible for
initializing test nodes, and the role of "testnode" is used by
testnodes.yml. "testnode" requires "qemu-system-x86" or "qemu-kvm"
package to be installed. the qemu in turn depends on librbd1 and
librados2.

before librados3 was introduced, this worked perfectly. because in ceph
repo, qa/packages/packages.yaml defines the default set of packages the
"install" tasks should install. and in that yaml file, librados2 was
listed. so the package management system will overwrite the librados2
installed by ansible playbook with the version specified by the
"install" task, as apt/yum thinks this is what user requires explicitly,
so it's fine to install a different version of librados2.

after librados3 was introduced, librados2 was removed from
qa/packages/packages.yaml. because, by default, we need to install
librados3 instead of librados2 for ready a nautilus cluster. but the
problem is, the packge list also applies to "install" tasks installing
releases before nautilus, where we still need to replace the librados2
installed by ansible.

so, to address this issue, "librados2" is added to "extra_packages" of
the "install" tasks of tests installing old releases to install
librados2 explicitly instead of as a dependency of other ceph packages
like librbd1.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25098 from ceph/rm-27062

ceph-volume: reject devices that have existing GPT headers

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #25093 from alfredodeza/wip-bz1644828

ceph-volume: remove LVs when using zap --destroy

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

os/tests: silence -Wsign-compare warning

silence warning like

In file included from
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/objectstore/store_test.cc:25:0:
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/googletest/googletest/include/gtest/gtest.h:
In instantiation of 'testing::AssertionResult
testing::internal::CmpHelperEQ(const char*, const char*, const T1&,
const T2&) [with T1 = int; T2 = long unsigned int]':

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-volume: update tests after removing the LVPath arg validator

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

ceph-volume: return lvs as vg/lv from ValidDevice if as_string is True

All code in prepare and activate expect lvs to be in that format.
Eventually we can refactor all that to use the Device class as well.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

ceph-volume: adds lv_name property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

ceph-volume: combine the LVPath arg validator with ValidDevice

This combines the two arg validators and makes both batch and
prepare/active both use ValidDevice.

This will also allow us to use lvs with a full path, like
/dev/vg/lv instead of always enforcing vg/lv.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

ceph-volume: adds has_gpt_headers property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

ceph-volume: add PTTYPE to blkid parser

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062

Merge pull request #25094 from CharlotteRose/wip-fix-boxes

doc: Fixed the paragraph and boxes

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #24789 from Ranjitha-G/dashboard-test-erasure

mgr/dashboard: Add unit test case for controller/erasure_code_profile.py

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: Fixed the paragraph and boxes.

Signed-off-by: Scoots Hamilton <scoots@redhat.com>

Merge pull request #24788 from Ranjitha-G/dashboard-hacking-doc

mgr/dashboard: Fix some setup steps in HACKING.rst

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #25080 from smithfarm/wip-37089

tests: make ceph-admin-commands.sh log what it does

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #25083 from iridescent-rsy/fix-typo

rbd: fix some typos

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd_mirror: fix a typo.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

rbd_mirror: restore the header annotation.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

rbd: fix typos.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

doc/releases/mimic.rst: make note of 13.2.2 upgrade bug

Signed-off-by: Neha Ojha <nojha@redhat.com>

ceph-volume tests update tests to handle new SystemExit exceptions in main

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.api verify that Volume object can be removed

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume api.lvm allow passing a Volume object to remove_lv

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.functional.lvm test full removal of LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume main raise sys.exit to prevent garbage in help

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.zap remove vgs|lvs when --destroy is used

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #24490 into master

* refs/pull/24490/head:
mds: flush dirty dirfrags that weren't logged when deactivating mds
mds: use MDlog::trim_all() to trim log when deactivating mds
mds: don't cap log when there are replicated objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

tests: make ceph-admin-commands.sh log what it does

Fixes: http://tracker.ceph.com/issues/37089
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25057 from joke-lee/wip-ceph-dencoder-support

ceph-dencoder: add RGWRealm and RGWPeriod  support.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25071 from tchaikov/wip-install-dep-for-python-saml

install-deps.sh: install libtool-ltdl-devel for building python-saml

Reviewed-by: Ricardo Marques <rimarques@suse.com>

install-deps.sh: install libtool-ltdl-devel for building python-saml

python-saml depends on dm.xmlsec.binding. which links against ltdl. so
without libtool-ltdl-devel, we will have

/opt/rh/devtoolset-7/root/usr/libexec/gcc/aarch64-redhat-linux/7/ld:
cannot find -lltdl

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dencoder: add RGWRealm support.

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge pull request #25037 from tchaikov/wip-qa-upgrade

qa: add librados3 to exclude_packages for ugprade tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #25063 from alfredodeza/wip-rm36768

ceph-volume: patch Device when testing

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests patch Device() by splitting parametrized method

This was causing failures on systems where there is no LVM or where the
device names don't match. Patching is always recommended to avoid
conflicts with the system testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests inventory should patch before using .get_devices()

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #25039 from tchaikov/wip-install-setuptools

install-deps: install setuptools before upgrading virtualenv

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #25051 into master

* refs/pull/25051/head:
doc/governance: reference the new Ceph Foundation

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

doc/governance: reference the new Ceph Foundation

Signed-off-by: Sage Weil <sage@redhat.com>

install-deps: install setuptools before upgrading virtualenv

this should address the failures when running install-deps.sh, like

Downloading/unpacking virtualenv
  Running setup.py egg_info for package virtualenv
    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
      warnings.warn(msg)
    error in virtualenv setup command: 'extras_require' must be a
dictionary whose values are strings or lists of strings containing valid
project/version requirement specifiers.
    Complete output from command python setup.py egg_info:
    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'

  warnings.warn(msg)

error in virtualenv setup command: 'extras_require' must be a dictionary
whose values are strings or lists of strings containing valid
project/version requirement specifiers.

this only happens on very old virtualenv shipped with RHEL7.4

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #25033 into master

* refs/pull/25033/head:
doc/governance: update CLT membership

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #25038 from yuriw/wip-yuriw-crontab-master

qa/tests: added "-n 7" to make sure mimic-x runs on built master branch

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tests: added "-n 7" to make sure mimic-x runs on built master branch

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc/governance: update CLT membership

- Neha is the new RADOS lead, but Josh is sticking around on the CLT
- John is leaving Red Hat and the Ceph project (we will miss him!)
- alphabetize

Signed-off-by: Sage Weil <sage@redhat.com>

qa: add librados3 to exclude_packages for ugprade tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #25023 into master

* refs/pull/25023/head:
doc/governance: Add Sebastian to CLT

Reviewed-by: Sage Weil <sage@redhat.com>

doc/governance: Add Sebastian to CLT

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #24830 from dillaman/wip-rbd-pool-stats

rbd: expose pool stats summary tool

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24265 from trociny/wip-36091

osd: collect client perf stats when query is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24214 from iridescent-rsy/bursting_io

librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22476 from dzafman/wip-23875

Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24806 into master

* refs/pull/24806/head:
os/filestore: fix merge_collection replay guards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24843 into master

* refs/pull/24843/head:
osd/osd_types: fix notify-ack string rendering

Reviewed-by: Neha Ojha <nojha@redhat.com>

rbd: added new 'pool stats' action

This provides a quick summary of provisioned pool usage.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new pool init/stat API methods

The init method is a stub for handling new pool initialization. It
currently only handles setting the application tag. The stats method
will quickly calculate the number of images and provisioned space for
those images within the pool. Querying the pool stats on a pool with
10,000 images only required approximately 2 seconds as compared to
over 2 minutes for a "rbd ls -l" scan.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Add unit test case for controller/erasure_code_profile.py

Signed-off-by: Ranjitha G <ranjitha.kmg@gmail.com>

Merge pull request #24998 from alfredodeza/wip-rm36470

ceph-volume enable  --no-systemd flag for simple sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24627 from Devp00l/wip-manage-ec-profiles

mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #24459 into master

* refs/pull/24459/head:
os: Transaction uses append_hole() to minimize bl:_buffers inflation.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #24890 into master

* refs/pull/24890/head:
mon: fix 'log last' missing out latest event

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24904 into master

* refs/pull/24904/head:
common: point of this may be called after released

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24925 into master

* refs/pull/24925/head:
Avoid import _strptime failed
Avoid exception if remote plugin not enabled
Separate diskprediction local plugin from the diskprediction plugin

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24859 from jan--f/ceph-volume-inventory

ceph-volume: add inventory command

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: add inventory command

The inventory command provides information about a nodes disk inventory.
Existing logical volumes on a disk or one of its partitions are scanned
and reported.
The output can be formatted as plain text or json.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #24934 from votdev/improve_crush_map_viewer

mgr/dashboard: Improve CRUSH map viewer

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24489 from ricardoasmarques/wip-saml2

mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Show info button for crush rules in pool form

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Adds ECP management to the frontend

Now you can create, delete and get information about profiles inside
the pool form.

The erasure code profile form has a lot of tooltips to guide you through
the creation. It can create profiles with different plugins.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Adds ECP info endpoint

The new info endpoint will provide the frontend with the necessary
information it needs to create new profiles.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #24928 from rhcs-dashboard/36632-update-python-dependency

mgr/dashboard: update python dependency

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24963 from dillaman/wip-ceph-iscsi

doc: tweak RBD iSCSI docs to point to merged tooling repo

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Fix some setup steps in HACKING.rst

Signed-off-by: Ranjitha G <ranjitha.kmg@gmail.com>

test: osd-scrub-snaps.sh: After snapshot removal wait for snaptrim to complete

Due to deliberate corruptions snaptrim_error means snaptrim is done

Signed-off-by: David Zafman <dzafman@redhat.com>

test: osd-scrub-snaps.sh: Testing with new --rmtype in ceph-objectstore-tool

Use --rmtype snapmap with new obj16 to remove snapmap only, check for repair message
Use --rmtype nosnapmap to remove obj5 while leaving snapmap behind

Signed-off-by: David Zafman <dzafman@redhat.com>

test: cleanup: Remove redundant cat of log and handle errors in create_scenario()

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-objectstore-tool: Add removal options to corrupt objects for testing

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #21511 into master

* refs/pull/21511/head:
os/bluefs: set logfile w/ WRITE_LIFE_MEDIUM
os/bluefs: make super block w/ WRITE_LIFE_TIME_SHORT.
os/bluestore: BlueFS support write_life_time feature of SSD.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24820 into master

* refs/pull/24820/head:
os/bluestore/BlueStore.cc: merge overlapping/adjacent regions before read

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24828 into master

* refs/pull/24828/head:
qa/osd-bluefs-volume-ops: use ceph-bluestore-tool for fsck
qa/osd-bluefs-volume-ops: reduce space usage for the test case

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #24948 into master

* refs/pull/24948/head:
os/bluestore: fix wal dir presence verification

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24014 from yehudasa/wip-rgw-svc-2

rgw: initial RGWRados refactoring work

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

rgw: shutdown services only if initialized

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

librados_test_stub: implement move operations

Need to match whatever librados implements.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph-volume tests ensure activate behavior with systemd disabling

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

ceph-volume simple.activate factor out systemd interactions for easier testing

Signed-off-by: Alfredo Deza <alfredo@deza.pe>

Merge PR #24995 into master

* refs/pull/24995/head:
doc: use 'proxy' cache mode in "Removing a Writeback Cache" procedure

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24896 from tchaikov/wip-librados-cleanup

librados,rpm,deb: various fixes to address librados3 transition and cleanups in librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: use 'proxy' cache mode in "Removing a Writeback Cache" procedure

The 'forward' mode is flawed as it allows the reordering of requests.

Signed-off-by: Mykola Golub <mgolub@suse.com>

test/rbd_mirror: fix linking

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: svc_notify: call set_enabled() when registering callback

svc_notify starts before svc_sysobj_cache, this way we make sure
svc_sysobj_cache gets the status callback triggered.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sysobj_core: filter sysobj attrs

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

librados: add move operations to IoCtx

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: chained cache shouldn't access service after it was unchained

This fixes a cleanup ordering issue. We can now have the chained cache
removed before and after the service is destructed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>