]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc: fix broken links multimds and kcephfs 39988/head
Jos Collin [Mon, 22 Feb 2021 15:08:18 +0000 (20:38 +0530)]
doc: fix broken links multimds and kcephfs

Fixes: https://tracker.ceph.com/issues/49372
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit efb23dd034460b66fc96cc5f891a23d1623841cb)

 Conflicts:
doc/dev/developer_guide/tests-integration-tests.rst
the documentation changed a lot but the links
referencing master are 404 and should be removed

4 years agoMerge pull request #39963 from neha-ojha/wip-49670-pacific
Neha Ojha [Wed, 10 Mar 2021 01:24:14 +0000 (17:24 -0800)]
Merge pull request #39963 from neha-ojha/wip-49670-pacific

pacific: src/test/smoke.sh: increase timeout

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #39921 from neha-ojha/wip-revert-autoscaler-pacific
Yuri Weinstein [Tue, 9 Mar 2021 23:58:59 +0000 (15:58 -0800)]
Merge pull request #39921 from neha-ojha/wip-revert-autoscaler-pacific

pacific: Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
4 years agoMerge pull request #39844 from neha-ojha/wip-49565-pacific
Yuri Weinstein [Tue, 9 Mar 2021 23:57:38 +0000 (15:57 -0800)]
Merge pull request #39844 from neha-ojha/wip-49565-pacific

pacific: tests: ceph_test_rados_api_watch_notify: Allow for reconnect

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agosrc/test/smoke.sh: increase timeout 39963/head
Sage Weil [Thu, 4 Mar 2021 23:34:00 +0000 (17:34 -0600)]
src/test/smoke.sh: increase timeout

Fixes: https://tracker.ceph.com/issues/47419
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 69d7aa9bf1b0de490e70cedd0463d3ca765a5673)

4 years agoMerge pull request #39901 from tchaikov/pr-39461-pacific
Yuri Weinstein [Tue, 9 Mar 2021 17:35:48 +0000 (09:35 -0800)]
Merge pull request #39901 from tchaikov/pr-39461-pacific

pacific: rgw/test: allow for duplicate events in pubsub

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
4 years agoMerge pull request #39829 from smithfarm/wip-49593-pacific
Yuri Weinstein [Tue, 9 Mar 2021 16:11:44 +0000 (08:11 -0800)]
Merge pull request #39829 from smithfarm/wip-49593-pacific

pacific: rpm: use PMDK system libraries on SUSE

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #39925 from tchaikov/pacific-github-workflow-milestone
Kefu Chai [Tue, 9 Mar 2021 09:21:03 +0000 (17:21 +0800)]
Merge pull request #39925 from tchaikov/pacific-github-workflow-milestone

pacific: .github: add workflow for adding milestone

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years ago.github/workflows: use a released sha1 39925/head
Kefu Chai [Tue, 9 Mar 2021 04:16:26 +0000 (12:16 +0800)]
.github/workflows: use a released sha1

otherwise the action cannot be found

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ae8ea10f415ec170be4a7be48567dbac83221dc0)

4 years ago.github/workflows: rename labeler.yml to pr-triage.yml
Kefu Chai [Mon, 8 Mar 2021 18:46:46 +0000 (02:46 +0800)]
.github/workflows: rename labeler.yml to pr-triage.yml

to reflect the change in the actions in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 453cd5fd602c54809dc091f0c3a498f903366c8f)

4 years ago.github/workflows: use @{sha1} for actions
Kefu Chai [Mon, 8 Mar 2021 18:26:34 +0000 (02:26 +0800)]
.github/workflows: use @{sha1} for actions

more secure this way. see also https://julienrenaux.fr/2019/12/20/github-actions-security-risk/

point the sha1 to

* labeler@v3
* milestone@main HEAD

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 79e8038046092053b0c0b120e0d7ca07a33a1c00)

4 years ago.github/workflows: move milestone.yml into labeler.yml
Kefu Chai [Sun, 7 Mar 2021 11:21:25 +0000 (19:21 +0800)]
.github/workflows: move milestone.yml into labeler.yml

no need to have two actions triggered by the same event.

we could backport this labeler.yml to LTS branches.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 98b90189791d3f4d2524fd829edb616d0ffd94f7)

4 years ago.github: correct the regex in mileston workflow
Kefu Chai [Sat, 6 Mar 2021 16:32:42 +0000 (00:32 +0800)]
.github: correct the regex in mileston workflow

also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c5f6d15e36fdd019196a1520525382f73f276f14)

4 years ago.github: add workflow for adding milestone
Kefu Chai [Sat, 6 Mar 2021 13:18:13 +0000 (21:18 +0800)]
.github: add workflow for adding milestone

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1657a44750442ff2582d601fb028aa89e3a95999)

4 years agoRevert "mgr/pg_autoscaler: avoid scale-down until there is pressure" 39921/head
Neha Ojha [Mon, 8 Mar 2021 17:16:15 +0000 (17:16 +0000)]
Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

This reverts commit de2a4684ea2e8719cc18a70d9f4be9715e409264.

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39741 from tchaikov/pacific-49502
Yuri Weinstein [Mon, 8 Mar 2021 16:08:30 +0000 (08:08 -0800)]
Merge pull request #39741 from tchaikov/pacific-49502

pacific: mgr/rbd_support: bail out if snapshot mirroring is not enabled

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
4 years agoMerge PR #39856 into pacific
Sage Weil [Mon, 8 Mar 2021 15:42:59 +0000 (09:42 -0600)]
Merge PR #39856 into pacific

* refs/pull/39856/head:
qa/distro/ubuntu_20.04_podman: Avoid getting asked
qa/suites/rados/cephadm: drop centos/rhel cephadm tests for the moment
qa/sites/rados/cephadm/thrash: rename 3-tasks.yaml/ -> 3-tasks/
qa/suites/rados/cephadm: adjust distros
qa/suites/upgrade: use kubic; test all distros
qa/suites/rados/cephadm/upgrade: use kubic on centos
qa: new kubic distro files; use kubic podman for centos/rhel
qa/suites/rados/cephadm: Add 20.04 podman:testing

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agocmake: make the linkage to pmem::pmemobj public 39829/head
Kefu Chai [Sat, 6 Mar 2021 07:43:33 +0000 (15:43 +0800)]
cmake: make the linkage to pmem::pmemobj public

tools/ceph-dencoder/rbd_types.cc includes Types.h which in turn includes
libpmemobj.h via librbd/cache/pwl/Types.h. and ceph-dencoder pulls in the
rbd_type.cc's linked libraries by linking against rbd_types. but before
this change, rbd_types links against pmem::pmemobj as a PRIVATE library.

so, if we want to pull in rbd_types linkage we should always link
rbd_types as a PUBLIC library. as rbd_types include libpmemobj.h in its
header file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a7e86a0d4463e5c1d4b614d7596df99eed4e42c5)

4 years agocmake: link libpmemobj against libpmem
Kefu Chai [Sat, 6 Mar 2021 04:22:39 +0000 (12:22 +0800)]
cmake: link libpmemobj against libpmem

libpmemobj should link against libpmem, but, in CMake, imported library
does not allow PRIVATE linkage. so pmem::pmem is added to the list of
INTERFACE_LINK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit aeb22b47a60cbc1133a0ab40ea505da52ba1d36e)

4 years agocmake: support COMPONENTS param in Findpmem.cmake
Kefu Chai [Fri, 5 Mar 2021 06:04:23 +0000 (14:04 +0800)]
cmake: support COMPONENTS param in Findpmem.cmake

add two components: pmem and pmemobj to this package. so we can find
them and link against them in a more intuitive way.

before this change the COMPONENTS parameter passed to

find_package(pmem ...)

is dropped on the floor and ignored.

after this change, it is checked and taken into consideration.

also, in this change, the exposed variables are renamed from

PMEM_* to pmem_*

to be consistent with the package name. it's encouraged to be consistent
with the package name when it comes to the INCLUDE_DIR and LIBRARIES
variable names.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 48e4cd190f65ed92fa9ca41b147621b46b4ea278)

4 years agocmake: do not link blk against pmem::pmemobj
Kefu Chai [Fri, 5 Mar 2021 13:15:59 +0000 (21:15 +0800)]
cmake: do not link blk against pmem::pmemobj

as blk does not use pmem::pmemobj, librbd does.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3e991ad1dbd8bf32175294500c38175bfb0038a7)

4 years agorgw/test: allow for duplicate events in pubsub 39901/head
Yuval Lifshitz [Sun, 14 Feb 2021 15:19:17 +0000 (17:19 +0200)]
rgw/test: allow for duplicate events in pubsub

when tested via radosgw-admin.
also, add verification for the content of the events

Fixes: https://tracker.ceph.com/issues/49261
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit a695cf7e9d8351f67f4ad210a4359e2067769b72)

4 years agoqa/distro/ubuntu_20.04_podman: Avoid getting asked 39856/head
Sebastian Wagner [Mon, 22 Feb 2021 13:58:38 +0000 (14:58 +0100)]
qa/distro/ubuntu_20.04_podman: Avoid getting asked

... for confic changes interactively

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0defae4e65dee123ee0b8474bf3cb417ddff7eb1)

4 years agoMerge PR #39628 into pacific
Sage Weil [Fri, 5 Mar 2021 20:35:26 +0000 (14:35 -0600)]
Merge PR #39628 into pacific

* refs/pull/39628/head:
mgr/dashboard: set security headers

Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge PR #39702 into pacific
Sage Weil [Fri, 5 Mar 2021 20:35:15 +0000 (14:35 -0600)]
Merge PR #39702 into pacific

* refs/pull/39702/head:
mon: simplify fs 'services' view of 'ceph status'
mds: add volumes + status to data: section of 'ceph status'

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge PR #39727 into pacific
Sage Weil [Fri, 5 Mar 2021 20:35:05 +0000 (14:35 -0600)]
Merge PR #39727 into pacific

* refs/pull/39727/head:
mgr/telemetry: check if 'ident' channel is active
qa/tasks/mgr/dashboard: identify cluster in telemetry test

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
4 years agoMerge PR #39788 into pacific
Sage Weil [Fri, 5 Mar 2021 20:34:51 +0000 (14:34 -0600)]
Merge PR #39788 into pacific

* refs/pull/39788/head:
mon/ConfigMonitor: make config changes via KVMonitor's pending set

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge PR #39805 into pacific
Sage Weil [Fri, 5 Mar 2021 20:34:28 +0000 (14:34 -0600)]
Merge PR #39805 into pacific

* refs/pull/39805/head:
src/global/signal_handler.h: fix preprocessor logic for alpine

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #39811 into pacific
Sage Weil [Fri, 5 Mar 2021 20:34:16 +0000 (14:34 -0600)]
Merge PR #39811 into pacific

* refs/pull/39811/head:
doc/rbd: clarify QCOW2 migration support

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #39806 from liewegas/pr-39665-pacific
Yuri Weinstein [Fri, 5 Mar 2021 17:16:46 +0000 (09:16 -0800)]
Merge pull request #39806 from liewegas/pr-39665-pacific

pacific: qa/rados/dashbord use "random" objectore

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39796 from dzafman/wip-48984-pacific
Yuri Weinstein [Fri, 5 Mar 2021 17:16:03 +0000 (09:16 -0800)]
Merge pull request #39796 from dzafman/wip-48984-pacific

pacific: osd: Don't clear m_planned_scrub until scrub finishes successfully

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39773 from myoungwon/pacific-backport-49482
Yuri Weinstein [Fri, 5 Mar 2021 17:15:32 +0000 (09:15 -0800)]
Merge pull request #39773 from myoungwon/pacific-backport-49482

pacific: osd, test: fix to avoid fail notification when testing manifest refcount

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #39753 from ifed01/wip-ifed-bluefs-zero-read-retry-pac
Yuri Weinstein [Fri, 5 Mar 2021 17:14:46 +0000 (09:14 -0800)]
Merge pull request #39753 from ifed01/wip-ifed-bluefs-zero-read-retry-pac

pacific: os/bluestore: Add option to check BlueFS reads

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
4 years agoMerge pull request #39748 from liewegas/pr-39631-pacific
Yuri Weinstein [Fri, 5 Mar 2021 17:14:20 +0000 (09:14 -0800)]
Merge pull request #39748 from liewegas/pr-39631-pacific

pacific: mon/OSDMonitor: do not mark newly created OSDs OUT

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39688 from ifed01/wip-ifed-fix-huge-bluefs-writes-pac
Yuri Weinstein [Fri, 5 Mar 2021 17:13:23 +0000 (09:13 -0800)]
Merge pull request #39688 from ifed01/wip-ifed-fix-huge-bluefs-writes-pac

pacific: os/bluestore: fix huge(>4GB) writes from RocksDB to BlueFS.

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoqa/suites/rados/cephadm: drop centos/rhel cephadm tests for the moment
Sage Weil [Thu, 4 Mar 2021 18:33:12 +0000 (12:33 -0600)]
qa/suites/rados/cephadm: drop centos/rhel cephadm tests for the moment

...until we sort out the podman problems!  :( :(

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e42223c4fc8ca42e03a5721ae5b0bc5362f144ce)

4 years agoqa/sites/rados/cephadm/thrash: rename 3-tasks.yaml/ -> 3-tasks/
Sage Weil [Thu, 4 Mar 2021 18:32:27 +0000 (12:32 -0600)]
qa/sites/rados/cephadm/thrash: rename 3-tasks.yaml/ -> 3-tasks/

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5f5472b7daede39a7bf35bf9135fc4fc75ac5a86)

4 years agoqa/suites/rados/cephadm: adjust distros
Sage Weil [Thu, 4 Mar 2021 18:32:00 +0000 (12:32 -0600)]
qa/suites/rados/cephadm: adjust distros

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 93e77ac3c81b5e6fc363786429b7eee0ac8567c3)

4 years agoqa/suites/upgrade: use kubic; test all distros
Sage Weil [Wed, 3 Mar 2021 14:23:56 +0000 (08:23 -0600)]
qa/suites/upgrade: use kubic; test all distros

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b378c042296858ec4d1c63a42cd257580ca11c96)

4 years agoqa/suites/rados/cephadm/upgrade: use kubic on centos
Sage Weil [Wed, 3 Mar 2021 14:18:19 +0000 (08:18 -0600)]
qa/suites/rados/cephadm/upgrade: use kubic on centos

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f4de8738ca86b7b31998718b53c8b7659fcdd27f)

4 years agoqa: new kubic distro files; use kubic podman for centos/rhel
Sage Weil [Wed, 3 Mar 2021 14:14:29 +0000 (08:14 -0600)]
qa: new kubic distro files; use kubic podman for centos/rhel

The current centos/rhel version of podman (2.2.1) is broken.

- create new qa/distros/podman/* files that install kubic podman
- include centos/rhel variants
- adjust cephadm jobs to use new yaml files
- remove old qa/distros/all/*_podman.yaml files

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 11beb7287084b56ac7cbd45ac3d28b3d9fa4cc5b)

4 years agoqa/suites/rados/cephadm: Add 20.04 podman:testing
Sebastian Wagner [Thu, 11 Feb 2021 15:01:57 +0000 (16:01 +0100)]
qa/suites/rados/cephadm: Add 20.04 podman:testing

make sure next podman versions aren't breaking cephadm

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 2216350b567102db4ab80dc42dfff9dced2b9740)

4 years agoMerge pull request #39808 from liewegas/pr-39801-pacific
Ernesto Puerta [Fri, 5 Mar 2021 09:20:43 +0000 (10:20 +0100)]
Merge pull request #39808 from liewegas/pr-39801-pacific

pacific: mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agotests: ceph_test_rados_api_watch_notify: Allow for reconnect 39844/head
Brad Hubbard [Mon, 22 Feb 2021 03:28:12 +0000 (13:28 +1000)]
tests: ceph_test_rados_api_watch_notify: Allow for reconnect

An injected socket failure may cause rados_watch_check() to return
ENOENT instead of the expected ENOTCONN.

Fixes: https://tracker.ceph.com/issues/47719
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 0a03a81f633f11bd3247b2f8f10f719c7b3d38e3)

4 years agoMerge PR #39737 into pacific
Sage Weil [Thu, 4 Mar 2021 21:08:22 +0000 (15:08 -0600)]
Merge PR #39737 into pacific

* refs/pull/39737/head:
mgr/DaemonServer: osd ok-to-stop: return json when there are unknown PGs
doc/man/8/ceph: document --max option
src/test/osd/safe-to-destroy: adjust test
ceph: print command output to stdout even on error
mgr/DaemonServer: include details in 'osd ok-to-stop' output
mgr: add --max <n> to 'osd ok-to-stop' command
mgr: relax osd ok-to-stop condition on degraded pgs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #39517 from k0ste/wip-49325-pacific
Ernesto Puerta [Thu, 4 Mar 2021 19:54:58 +0000 (20:54 +0100)]
Merge pull request #39517 from k0ste/wip-49325-pacific

pacific: mgr/dashboard: fix MTU Mismatch alert

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge PR #39736 into pacific
Sage Weil [Thu, 4 Mar 2021 18:49:37 +0000 (12:49 -0600)]
Merge PR #39736 into pacific

* refs/pull/39736/head:
crush/CrushWrapper: rebuild shadow tree on 'osd crush reweight-subtree'
crush/CrushWrapper: update shadow trees on update_item()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #39734 into pacific
Sage Weil [Thu, 4 Mar 2021 18:49:12 +0000 (12:49 -0600)]
Merge PR #39734 into pacific

* refs/pull/39734/head:
mon/PGMap: include the per-pool usage breakdown in pg dump

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #39735 into pacific
Sage Weil [Thu, 4 Mar 2021 18:46:41 +0000 (12:46 -0600)]
Merge PR #39735 into pacific

* refs/pull/39735/head:
mon/OSDMonitor: fix safety/idempotency of {set,rm}-device-class

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
4 years agoMerge PR #39597 into pacific
Sage Weil [Thu, 4 Mar 2021 18:46:18 +0000 (12:46 -0600)]
Merge PR #39597 into pacific

* refs/pull/39597/head:
osd: Don't clear m_planned_scrub until scrub finishes successfully

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agomgr/DaemonServer: osd ok-to-stop: return json when there are unknown PGs 39737/head
Sage Weil [Thu, 4 Mar 2021 13:35:24 +0000 (08:35 -0500)]
mgr/DaemonServer: osd ok-to-stop: return json when there are unknown PGs

In 791952cc01201010f298033003ba52374cc0159f we switched to return JSON
both on success and fail to describe which PGs are affected or are blocking
the ability to stop/restart OSDs.  Do the same for the case where
some PG states are unknown (i.e., just after a mgr restart) so that
the cephadm upgrade process can unconditionally expect a JSON result.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2cce16537c9f9e92bf0a8fce746d0364cbb2d15d)

4 years agoMerge pull request #39524 from rhcs-dashboard/wip-49326-pacific
Ernesto Puerta [Thu, 4 Mar 2021 12:54:36 +0000 (13:54 +0100)]
Merge pull request #39524 from rhcs-dashboard/wip-49326-pacific

pacific: mgr/dashboard: avoid using document.write()

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #39647 from rhcs-dashboard/wip-49390-pacific
Ernesto Puerta [Thu, 4 Mar 2021 12:53:54 +0000 (13:53 +0100)]
Merge pull request #39647 from rhcs-dashboard/wip-49390-pacific

pacific: mgr/dashboard: fix tooltip for Provisioned/Total Provisioned fields

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agorpm: use PMDK system libraries on SUSE
Nathan Cutler [Mon, 1 Mar 2021 11:07:29 +0000 (12:07 +0100)]
rpm: use PMDK system libraries on SUSE

As of a49d1dbb32e2436ff2836a85b2fa84418f0a5fff, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to

    git clone https://github.com/ceph/pmdk.git

but of course that won't work in the OBS, where the build workers have no
Internet connectivity.

Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.

Fixes: a49d1dbb32e2436ff2836a85b2fa84418f0a5fff
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 30df650e6496e35c798eab7d85ea0806fd58722b)

Conflicts:
ceph.spec.in

4 years agoMerge pull request #39816 from cbodley/wip-49585
Casey Bodley [Wed, 3 Mar 2021 20:59:01 +0000 (15:59 -0500)]
Merge pull request #39816 from cbodley/wip-49585

pacific: valgrind: suppress operator delete[] for tcmalloc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agovalgrind: suppress operator delete[] for tcmalloc 39816/head
Casey Bodley [Tue, 2 Mar 2021 14:31:08 +0000 (09:31 -0500)]
valgrind: suppress operator delete[] for tcmalloc

suppresses a mismatched operator delete[] from the librados client's
destructor:

<error>
  <unique>0xeef7eb</unique>
  <tid>1</tid>
  <kind>MismatchedFree</kind>
  <what>Mismatched free() / delete / delete []</what>
  <stack>
    <frame>
      <ip>0x4C32EA0</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator delete[](void*, unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>660</line>
    </frame>
    <frame>
      <ip>0x5FBEB86</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5FBEF2C</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5F7F7C5</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
      <fn>librados::v14_2_0::Rados::shutdown()</fn>
    </frame>

Fixes: https://tracker.ceph.com/issues/49501
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit bf087c3f840216a704600bb2436c643159421ea2)

4 years agomgr/dashboard: fix issues related with PyJWT versions >=2.0.0 39808/head
Alfonso Martínez [Wed, 3 Mar 2021 14:36:06 +0000 (15:36 +0100)]
mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Fixes: https://tracker.ceph.com/issues/49574
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 4b96bb51e8f133badd7bc651bcb4dcd755b43d75)

4 years agodoc/rbd: clarify QCOW2 migration support 39811/head
Jason Dillaman [Tue, 2 Mar 2021 22:29:29 +0000 (17:29 -0500)]
doc/rbd: clarify QCOW2 migration support

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 68fda590cf88e6b232dfb573236fe3ae3949db28)

4 years agoMerge pull request #39568 from tchaikov/pr-39365-pacific
Kefu Chai [Wed, 3 Mar 2021 14:07:08 +0000 (22:07 +0800)]
Merge pull request #39568 from tchaikov/pr-39365-pacific

pacific: mgr/devicehaelth: extract and present SSD wear

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoqa/rados/dashbord use "random" objectore 39806/head
Sebastian Wagner [Wed, 24 Feb 2021 12:01:54 +0000 (13:01 +0100)]
qa/rados/dashbord use "random" objectore

1. removed symlink to `.qa/objectstore`
2. added `random-objectstore&` to `.qa/objectstore`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 72229a098b878036e2738147814898bc65efcaea)

4 years agosrc/global/signal_handler.h: fix preprocessor logic for alpine 39805/head
Duncan Bellamy [Thu, 25 Feb 2021 12:05:56 +0000 (12:05 +0000)]
src/global/signal_handler.h: fix preprocessor logic for alpine
Fixes: https://tracker.ceph.com/issues/49494
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
(cherry picked from commit 387f33e7265825ae4cdd3580289f5154e250b05b)

4 years agoMerge PR #39525 into pacific
Jan Fajerski [Wed, 3 Mar 2021 08:40:13 +0000 (09:40 +0100)]
Merge PR #39525 into pacific

* refs/pull/39525/head:
ceph-volume: pass --filter-for-batch from drive-group subcommand

Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoMerge PR #39468 into pacific
Jan Fajerski [Wed, 3 Mar 2021 08:31:24 +0000 (09:31 +0100)]
Merge PR #39468 into pacific

* refs/pull/39468/head:
Update batch.py

Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoqa/tasks/mgr/dashboard: add wear_level to devices schema 39568/head
Sage Weil [Sun, 21 Feb 2021 16:40:06 +0000 (10:40 -0600)]
qa/tasks/mgr/dashboard: add wear_level to devices schema

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6f6792cf51f9042452bc79df30b3a4d02dd5ef96)

4 years agomgr/devicehealth: extract+store wear level from metrics scraping
Sage Weil [Thu, 18 Feb 2021 14:27:49 +0000 (08:27 -0600)]
mgr/devicehealth: extract+store wear level from metrics scraping

When we scrape and store health metrics for a device, extract the wear
level from the JSON.  If present, also store it in the config-key
per-device metadata.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4840507cfcdd5182003671994d0bc9604d072e3e)

 Conflicts:
src/pybind/mgr/devicehealth/module.py
 - type annotations for _get_device_metrics

4 years agomgr/DaemonServer: show wear level in 'device ls' output
Sage Weil [Mon, 8 Feb 2021 18:54:04 +0000 (12:54 -0600)]
mgr/DaemonServer: show wear level in 'device ls' output

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8f93e3b55351df0bc8308e0cace482aabb15e8c8)

4 years agomgr/DaemonState: store optional wear_level for devices
Sage Weil [Mon, 8 Feb 2021 18:53:51 +0000 (12:53 -0600)]
mgr/DaemonState: store optional wear_level for devices

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c52fcb51afb6da2dccd6954a40f0f1c616324855)

4 years agocommon/blkdev: collect non-SMART data too
Sage Weil [Mon, 8 Feb 2021 18:53:24 +0000 (12:53 -0600)]
common/blkdev: collect non-SMART data too

Call smartctl with -x instead of -a:

       -a, --all
              Prints  all SMART information about the disk, or TapeAlert infor‐
              mation about the tape drive or changer.  For ATA devices this  is
              equivalent to
              '-H -i -c -A -l error -l selftest -l selective'
              and for SCSI, this is equivalent to
              '-H -i -A -l error -l selftest'.
              For NVMe, this is equivalent to
              '-H -i -c -A -l error'.
              Note  that  for  ATA  disks  this  does  not enable the non-SMART
              options and the SMART options which require  support  for  48-bit
              ATA commands.

vs

       -x, --xall
              Prints all SMART and non-SMART information about the device.  For
              ATA devices this is equivalent to
              '-H -i -g all -g wcreorder -c -A  -f  brief  -l  xerror,error  -l
              xselftest,selftest -l selective -l directory -l scttemp -l scterc
              -l devstat -l defects -l sataphy'.
              and for SCSI, this is equivalent to
              '-H -i -g all -A -l error -l selftest -l background -l sasphy'.
              For NVMe, this is equivalent to
              '-H -i -c -A -l error'.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 176bb54da90e404646c0e80e0c3ea804bb8a58d6)

4 years agomgr/DaemonState: fix serialization/storage of device life expectancy min/max
Sage Weil [Mon, 8 Feb 2021 16:41:48 +0000 (10:41 -0600)]
mgr/DaemonState: fix serialization/storage of device life expectancy min/max

These weren't encoded as strings, which means they were effectively
forgotton on each mgr restart.  Sigh.

Fixes: https://tracker.ceph.com/issues/49215
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 78b72b0be752fce855203b28b558133a0d47eba8)

4 years agopybind/mgr: silence flake8 warnings
Kefu Chai [Fri, 29 Jan 2021 02:30:19 +0000 (10:30 +0800)]
pybind/mgr: silence flake8 warnings

silence following warnings from flake8

* line too long (107 > 100 characters) [E501]

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 367f264100778717890b7fb69668e44f0a22d8ce)

4 years agopybind/mgr/mgr_module: correct annotation
Kefu Chai [Wed, 27 Jan 2021 04:07:07 +0000 (12:07 +0800)]
pybind/mgr/mgr_module: correct annotation

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 19cdc4325ce47bef951fdab41fa15e6acadea1f7)

4 years agomgr/devicehealth: remove handle_command()
Kefu Chai [Wed, 27 Jan 2021 06:34:35 +0000 (14:34 +0800)]
mgr/devicehealth: remove handle_command()

as commands are collected by COMMANDS already.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1361b3060e7643a3a0f2f59b72eb5d9201c21bcd)

4 years agomgr/devicehealth: return -1 if prediction plugin is not available
Kefu Chai [Wed, 27 Jan 2021 06:58:01 +0000 (14:58 +0800)]
mgr/devicehealth: return -1 if prediction plugin is not available

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bb611c64ee5120272215683b4b5ac4397f9c5ea6)

4 years agomgr/devicehealth: add more annotations
Kefu Chai [Wed, 27 Jan 2021 04:08:11 +0000 (12:08 +0800)]
mgr/devicehealth: add more annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 48438cccedb383e19cab743dd0222d6ec6265588)

4 years agoosd: Don't clear m_planned_scrub until scrub finishes successfully 39796/head
David Zafman [Wed, 3 Mar 2021 00:24:53 +0000 (16:24 -0800)]
osd: Don't clear m_planned_scrub until scrub finishes successfully

Fixes: https://tracker.ceph.com/issues/48984
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0b63c850a251f75bcb7436efefe4f24480c4ed7e)

4 years agoMerge pull request #39769 from ivancich/wip-49185-pacific
J. Eric Ivancich [Tue, 2 Mar 2021 21:07:17 +0000 (16:07 -0500)]
Merge pull request #39769 from ivancich/wip-49185-pacific

pacific: rgw: tooling to locate rgw objects with missing rados components

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
4 years agoMerge pull request #39764 from ivancich/wip-49194-pacific
J. Eric Ivancich [Tue, 2 Mar 2021 21:06:31 +0000 (16:06 -0500)]
Merge pull request #39764 from ivancich/wip-49194-pacific

pacific: rgw: allow rgw-orphan-list to handle intermediate files w/ binary data

Reviewed-by: Thomas Serlin <tserlin@redhat.com>
4 years agoMerge pull request #39675 from dillaman/wip-librbd-backports-pacific-6
Jason Dillaman [Tue, 2 Mar 2021 20:06:04 +0000 (15:06 -0500)]
Merge pull request #39675 from dillaman/wip-librbd-backports-pacific-6

pacific: librbd: miscellaneous backports

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #39635 from vshankar/wip-49425
Yuri Weinstein [Tue, 2 Mar 2021 19:32:13 +0000 (11:32 -0800)]
Merge pull request #39635 from vshankar/wip-49425

pacific: cephfs-mirror: run mirror daemon with valgrind

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #39508 from k0ste/wip-49315-pacific
Yuri Weinstein [Tue, 2 Mar 2021 19:01:19 +0000 (11:01 -0800)]
Merge pull request #39508 from k0ste/wip-49315-pacific

pacific: mgr/dashboard: prometheus alerting: add some leeway for package drops and errors

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
4 years agoMerge pull request #39466 from petrutlucian94/wip-49250-pacific
Yuri Weinstein [Tue, 2 Mar 2021 19:00:20 +0000 (11:00 -0800)]
Merge pull request #39466 from petrutlucian94/wip-49250-pacific

pacific: common: Add Windows ETW support

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomon/ConfigMonitor: make config changes via KVMonitor's pending set 39788/head
Sage Weil [Fri, 26 Feb 2021 16:42:52 +0000 (11:42 -0500)]
mon/ConfigMonitor: make config changes via KVMonitor's pending set

We need to ensure that changes we make to the kv store (config/...)
are proposed via KVMonitor so that they are properly versioned there
and shared with subscribers (notably, the mgr).

Fixes: bb7ebc41532aeb23cff2241ab07b3f01c2f57ddd
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dab72abd0ae8a3038f73dbe0983b2eaef3937ef6)

4 years agoMerge pull request #39752 from rhcs-dashboard/wip-49546-pacific
Ernesto Puerta [Tue, 2 Mar 2021 10:37:18 +0000 (11:37 +0100)]
Merge pull request #39752 from rhcs-dashboard/wip-49546-pacific

pacific: mgr/dashboard: Splitting tenant$user when creating rgw user

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
4 years agoosd, test: wait if the snapshot is deleting 39773/head
myoungwon oh [Tue, 16 Feb 2021 05:42:44 +0000 (14:42 +0900)]
osd, test: wait if the snapshot is deleting

After calling selfmanaged_snap_remove, we don't know
when trimming snapshot is finished.
So, we make the OSD to return EBUSY if the snapshot in removed_snap_queue,
then the unit test waits the completion

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit d6f9f23012b9ac133e767bd07a804707ba2207ef)

4 years agosrc/test: fix to avoid fail notification when testing manifest refcount
myoungwon oh [Mon, 18 Jan 2021 03:16:32 +0000 (12:16 +0900)]
src/test: fix to avoid fail notification when testing manifest refcount

Due to false-positive design on manifest snap refcounting,
a message to decrement the refcount can be missing.
This commit checks whether the manifest object's state is correct
when such mismatch happens to prevent aborting unit test.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit d0369dc73ee9a911e39642f3b63c4d0f17b04ce3)

4 years agoosd: remove unnecessary check
myoungwon oh [Fri, 15 Jan 2021 09:38:12 +0000 (18:38 +0900)]
osd: remove unnecessary check

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit 1109a59e2941cf4b16798cd04941daf350ec0d22)

4 years agoosd, test: return the number of references of chunked object
myoungwon oh [Fri, 8 Jan 2021 06:17:16 +0000 (15:17 +0900)]
osd, test: return the number of references of chunked object

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit cb7cf5ba545386e1bf010d655f0736d0ef5a29ae)

4 years agotest: add cls_has_chunk test to count chunk in snapshot
myoungwon oh [Tue, 5 Jan 2021 12:47:37 +0000 (21:47 +0900)]
test: add cls_has_chunk test to count chunk in snapshot

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit 11b52b4d00c7d56f0a571fd9208740932b0d00dd)

4 years agoosd: add has_manifest_chunk() to check chunks in snapshot
myoungwon oh [Tue, 5 Jan 2021 12:40:36 +0000 (21:40 +0900)]
osd: add has_manifest_chunk() to check chunks in snapshot

cls_has_chunk does not cover snapshotted manifest object.
This leads to unexpected behavior during chunk scrub.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry-picked from commit bdcedd4e9f2efed59b1e5c2d40171e6da595a259)

4 years agorgw: add rgw-gap-list-comparator tool 39769/head
J. Eric Ivancich [Fri, 29 Jan 2021 17:03:50 +0000 (12:03 -0500)]
rgw: add rgw-gap-list-comparator tool

The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.

Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
(cherry picked from commit 3c5154f010a34f9845b40e49a7870771e7f5372e)

4 years agorgw: add rgw-gap-list tool
J. Eric Ivancich [Thu, 17 Dec 2020 23:21:36 +0000 (18:21 -0500)]
rgw: add rgw-gap-list tool

Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.

IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
(cherry picked from commit 07b42195fbbcd27e330cb1daa35e77e0952f8a3c)

4 years agorgw: add radosgw-admin bucket radoslist bucket/obj mapping
J. Eric Ivancich [Thu, 17 Dec 2020 23:32:20 +0000 (18:32 -0500)]
rgw: add radosgw-admin bucket radoslist bucket/obj mapping

A new command-line option "--rgw-obj-fs" is added to
radosgw-admin. When used with the "bucket radoslist" subcommand, will
output lines with a rados object, bucket name, and object name,
separated by the field separator specified. Without this command-line
option, only the rados object is output, which is the previous
behavior.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 81d6260c1a42400bfa8c677c419e2c3f76d7a920)

4 years agorgw: allow rgw-orphan-list to handle intermediate files w/ binary data 39764/head
J. Eric Ivancich [Thu, 4 Feb 2021 21:19:17 +0000 (16:19 -0500)]
rgw: allow rgw-orphan-list to handle intermediate files w/ binary data

Updates the grep commands with the --text command-line option since
there are cases where rados object names contain NUL characters.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit b520cc5a910ea4fba0d2023b13e22b4166f767c7)

4 years agotest/librbd: resolve compile error on centos 39675/head
YuanXin [Sat, 27 Feb 2021 04:01:40 +0000 (12:01 +0800)]
test/librbd: resolve compile error on centos

Signed-off-by: YuanXin <yuanxin@didiglobal.com>
(cherry picked from commit 49a63a594038f96db235d20e82bcda9963f8ded8)

4 years agorbd: use portable zero-ing memory function
YuanXin [Sat, 27 Feb 2021 03:37:19 +0000 (11:37 +0800)]
rbd: use portable zero-ing memory function

Signed-off-by: YuanXin <yuanxin@didiglobal.com>
(cherry picked from commit 72c04a9d319520c29dfdf952408beb12ed73a88e)

4 years agoMerge pull request #39188 from liewegas/pr-39173-pacific
Yuri Weinstein [Mon, 1 Mar 2021 18:19:59 +0000 (10:19 -0800)]
Merge pull request #39188 from liewegas/pr-39173-pacific

pacific: mgr/devicehealth: do not create pool until there are enough osds

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39189 from liewegas/pr-39172-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:37:56 +0000 (09:37 -0800)]
Merge pull request #39189 from liewegas/pr-39172-pacific

pacific: mgr/orchestrator: remove 'host' arg for 'orch ls'

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39236 from k0ste/wip-49108-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:36:25 +0000 (09:36 -0800)]
Merge pull request #39236 from k0ste/wip-49108-pacific

pacific: mgr/zabbix: format ceph.[{#POOL},percent_used as float

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39663 from pponnuvel/wip-49442-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:21:12 +0000 (09:21 -0800)]
Merge pull request #39663 from pponnuvel/wip-49442-pacific

pacific: rgw: radosgw-admin: clarify error when email address already in use

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #39048 from smithfarm/wip-48971-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:20:33 +0000 (09:20 -0800)]
Merge pull request #39048 from smithfarm/wip-48971-pacific

pacific: rgw: Initialize Layout Logs

Reviewed-by: Casey Bodley <cbodley@redhat.com>