]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge PR #39848 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:23:13 +0000 (20:23 -0700)]
Merge PR #39848 into master

* refs/pull/39848/head:
mds: fix race of fetching large dirfrag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
4 years agoMerge PR #39847 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:21:52 +0000 (20:21 -0700)]
Merge PR #39847 into master

* refs/pull/39847/head:
qa/cephfs: don't expect "file exists" error for "mkdir -p"
qa/cephfs: don't take parameter cwd in _create_mntpt()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge pull request #40162 from liewegas/bug-49846
Kefu Chai [Wed, 17 Mar 2021 02:52:41 +0000 (10:52 +0800)]
Merge pull request #40162 from liewegas/bug-49846

mgr/prometheus: fix typo in get_collect_time_metrics

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40152 from TheJJ/fix-osd-class-path-docs
Kefu Chai [Wed, 17 Mar 2021 02:46:19 +0000 (10:46 +0800)]
Merge pull request #40152 from TheJJ/fix-osd-class-path-docs

doc: update osd-class-path error messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #40164 from racpatel/DocRBDfix
Jason Dillaman [Wed, 17 Mar 2021 00:21:37 +0000 (20:21 -0400)]
Merge pull request #40164 from racpatel/DocRBDfix

doc/RBD:fixes for ceph-immutable-object-cache daemon enable command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agodoc/RBD:fixes for ceph-immutable-object-cache daemon enable command 40164/head
Rachanaben Patel [Tue, 16 Mar 2021 22:37:46 +0000 (15:37 -0700)]
doc/RBD:fixes for ceph-immutable-object-cache daemon enable command

Document for rbd-persistent-read-only-cache show how to manage
ceph-immutable-object-cache daemon using systemd.
command example needs fixing.It should be

systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}

Fixes: https://tracker.ceph.com/issues/49849
Signed-off-by: Rachanaben Patel <racpatel@redhat.com>
4 years agomgr/prometheus: fix typo in get_collect_time_metrics 40162/head
Sage Weil [Tue, 16 Mar 2021 20:10:42 +0000 (15:10 -0500)]
mgr/prometheus: fix typo in get_collect_time_metrics

This causes a failure the first time through this function, but
subsequent calls succeed, making it a bit hard to notice.

Fixes: 58fd057e2c8799fa000b9937aa992e13cbbd485f
Fixes: https://tracker.ceph.com/issues/49846
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40114 into master
Patrick Donnelly [Tue, 16 Mar 2021 19:27:43 +0000 (12:27 -0700)]
Merge PR #40114 into master

* refs/pull/40114/head:
doc: reorganize Windows docs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge PR #40075 into master
Sage Weil [Tue, 16 Mar 2021 18:07:51 +0000 (14:07 -0400)]
Merge PR #40075 into master

* refs/pull/40075/head:
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #39931 into master
Sage Weil [Tue, 16 Mar 2021 17:04:55 +0000 (13:04 -0400)]
Merge PR #39931 into master

* refs/pull/39931/head:
mgr/cephadm: fall back to service spec port if none on DaemonDescription
mgr/cephadm: fix redeploy when daemons have ip:port
mgr/cephadm/schedule: add test case
qa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports
doc/cephadm/rgw: update docs to show count-per-host
mgr/cephadm: add support for rgw_frontend_type (beast or civetweb)
mgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec
mgr/cephadm: fix beast private key config option
mgr/cephadm: fix rgw ssl cert/key config-key path
mgr/cephadm/schedule: dynamically assign ports for rgw
mgr/cephadm/schedule: only 1 port in DaemonPlacement
mgr/cephadm: move rgw frontend logic into RgwService
mgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec
mgr/cephadm/schedule: remove unused methods
mgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment
cephadm: populate ports if known and not included in unit.meta
mgr/cephadm: gather and report ports in 'orch ps' output

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge pull request #40120 from zdover23/wip-doc-cephadm-mon-service-1-16-Mar-2021
zdover23 [Tue, 16 Mar 2021 16:44:33 +0000 (02:44 +1000)]
Merge pull request #40120 from zdover23/wip-doc-cephadm-mon-service-1-16-Mar-2021

doc/cephadm: break mon section into sections

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agodoc/cephadm: break mon section into sections 40120/head
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: break mon section into sections

This PR breaks the "Deploy Additional Monitors" section
of the cephadm documentation into several subsections
whose titles spotlight the matter under discussion in
those respective subsections.

inb4: Another PR is on deck that rewrites the sentences
in this chapter of the cephadm documentation. I'd like
to get this chapter broken up into these subsections before
I rewrite those sentences. So I'm hoping for no grammatical
mission creep on this one. The grammar and clarity updates
are coming.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc: update osd-class-path error messages 40152/head
Jonas Jelten [Tue, 16 Mar 2021 10:45:50 +0000 (11:45 +0100)]
doc: update osd-class-path error messages

the return value is "95 Operation not supported"

Signed-off-by: Jonas Jelten <jj@sft.lol>
4 years agoMerge pull request #40141 from zdover23/wip-doc-cephadm-mon-deploying-add-rewrite...
zdover23 [Tue, 16 Mar 2021 15:10:01 +0000 (01:10 +1000)]
Merge pull request #40141 from zdover23/wip-doc-cephadm-mon-deploying-add-rewrite-2021-Mar-16

doc/cephadm: rewrite "deploying add. mons"

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge pull request #40153 from rzarzynski/wip-osd-obclock-cleanup
Kefu Chai [Tue, 16 Mar 2021 14:04:32 +0000 (22:04 +0800)]
Merge pull request #40153 from rzarzynski/wip-osd-obclock-cleanup

osd: clean obc locking and repop creation as a sidequest of crimson's InternalClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39996 from tchaikov/wip-crimson-cleanups
Kefu Chai [Tue, 16 Mar 2021 14:01:58 +0000 (22:01 +0800)]
Merge pull request #39996 from tchaikov/wip-crimson-cleanups

crimson: cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoosd: move PrimaryLogPG::get_rw_locks() from header to .cc. 40153/head
Radoslaw Zarzynski [Tue, 16 Mar 2021 12:45:06 +0000 (12:45 +0000)]
osd: move PrimaryLogPG::get_rw_locks() from header to .cc.

For the sake of symmetry with `release_object_locks()`
and easier grepping for e.g. `ctx->lock_manager`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoosd: PrimaryLogPG::new_repop() doesn't depend on obc anymore.
Radoslaw Zarzynski [Mon, 15 Mar 2021 19:14:44 +0000 (19:14 +0000)]
osd: PrimaryLogPG::new_repop() doesn't depend on obc anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge PR #40130 into master
Sage Weil [Tue, 16 Mar 2021 12:43:06 +0000 (08:43 -0400)]
Merge PR #40130 into master

* refs/pull/40130/head:
mgr/cephadm: less noise about refreshing hosts

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge PR #40140 into master
Sage Weil [Tue, 16 Mar 2021 12:42:46 +0000 (08:42 -0400)]
Merge PR #40140 into master

* refs/pull/40140/head:
mgr/cephadm: fix 'auth caps' fallback

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agocrimson/os: do not include unused headers 39996/head
Kefu Chai [Wed, 10 Mar 2021 09:13:22 +0000 (17:13 +0800)]
crimson/os: do not include unused headers

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson: use GTEST_FAIL() for printing failure message
Kefu Chai [Wed, 10 Mar 2021 09:07:25 +0000 (17:07 +0800)]
test/crimson: use GTEST_FAIL() for printing failure message

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: drop duplicated __attribute__
Kefu Chai [Wed, 10 Mar 2021 09:07:02 +0000 (17:07 +0800)]
crimson/os: drop duplicated __attribute__

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/cephadm: rewrite "deploying add. mons" 40141/head
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: rewrite "deploying add. mons"

This rewrites the section "Deploying Additional
Monitors (Beyond the Default Three)" for elegance
and clarity.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agomgr/cephadm: fall back to service spec port if none on DaemonDescription 39931/head
Sage Weil [Mon, 15 Mar 2021 22:58:15 +0000 (18:58 -0400)]
mgr/cephadm: fall back to service spec port if none on DaemonDescription

For an RGW instance that is deployed in an older version, we won't have
IP or port metadata in the DaemonDescription.  In that case, fall back
to the port in the ServiceSpec.  This should be safe since any such
instance will only have 1 daemon per host.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix redeploy when daemons have ip:port
Sage Weil [Mon, 15 Mar 2021 19:34:13 +0000 (15:34 -0400)]
mgr/cephadm: fix redeploy when daemons have ip:port

The _daemon_action() method can be called directly by upgrade and by
the 'orch daemon <action> <name>' commands.  When this happens, construct
a CephadmDaemonDeploySpec from the DaemonDescription that incldes the
metadata we assigned when teh service was created: the IP and port(s).
This fixes upgrade and the CLI.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: add test case
Sage Weil [Thu, 11 Mar 2021 18:47:37 +0000 (13:47 -0500)]
mgr/cephadm/schedule: add test case

Matches wonky cephadm.py placement (+1 on the count).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports
Sage Weil [Wed, 10 Mar 2021 21:54:28 +0000 (16:54 -0500)]
qa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agodoc/cephadm/rgw: update docs to show count-per-host
Sage Weil [Wed, 10 Mar 2021 21:48:58 +0000 (16:48 -0500)]
doc/cephadm/rgw: update docs to show count-per-host

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: add support for rgw_frontend_type (beast or civetweb)
Sage Weil [Wed, 10 Mar 2021 22:51:49 +0000 (17:51 -0500)]
mgr/cephadm: add support for rgw_frontend_type (beast or civetweb)

Previously we were assuming beast.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec
Sage Weil [Wed, 10 Mar 2021 19:58:09 +0000 (14:58 -0500)]
mgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec

Since this didn't work anyway, stop collecting and passing through the
private key portion of the certificate.  Instead, users should include
both in the first option.  This is simpler, and provides consistency
across civetweb and beast rgw backends (for whatever that is worth).

NOTE: dashboard changes are not included here.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix beast private key config option
Sage Weil [Wed, 10 Mar 2021 19:50:30 +0000 (14:50 -0500)]
mgr/cephadm: fix beast private key config option

This has always been broken.  However, beast SSL will also accept a pem
(cert + key) via the ssl_certificate option, so any existing non-broken
users (if they exist) must be using that.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix rgw ssl cert/key config-key path
Sage Weil [Wed, 10 Mar 2021 22:50:22 +0000 (17:50 -0500)]
mgr/cephadm: fix rgw ssl cert/key config-key path

I broke this when making the realm and zone optional.

Fixes: af0216d93fb522d239b10b5d7d18208ee02fed77
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: dynamically assign ports for rgw
Sage Weil [Wed, 10 Mar 2021 19:25:23 +0000 (14:25 -0500)]
mgr/cephadm/schedule: dynamically assign ports for rgw

Dynamically number ports for RGW instances, with the start port being
the one configured on the service (or the default of 80 or 443).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: only 1 port in DaemonPlacement
Sage Weil [Wed, 10 Mar 2021 18:59:18 +0000 (13:59 -0500)]
mgr/cephadm/schedule: only 1 port in DaemonPlacement

It would be weird to dynamically number multiple ports (although doable).
But until we have plans to support something like that, no need to handle
it here.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: move rgw frontend logic into RgwService
Sage Weil [Wed, 10 Mar 2021 18:48:10 +0000 (13:48 -0500)]
mgr/cephadm: move rgw frontend logic into RgwService

This doesn't belong in the ServiceSpec/RGWSpec

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec
Sage Weil [Wed, 10 Mar 2021 17:24:32 +0000 (12:24 -0500)]
mgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec

Create a new type for the result of scheduling/place().  Include new fields
like ip:port.  Introduce a matches_daemon() to see whether an existing
daemon matches a scheduling slot.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: remove unused methods
Sage Weil [Wed, 10 Mar 2021 16:00:04 +0000 (11:00 -0500)]
mgr/cephadm/schedule: remove unused methods

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment
Sage Weil [Mon, 8 Mar 2021 21:07:20 +0000 (16:07 -0500)]
mgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment

Pass the port through to the CephadmDaemonDeploySpec, and unit.meta.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocephadm: populate ports if known and not included in unit.meta
Sage Weil [Mon, 8 Mar 2021 19:18:04 +0000 (14:18 -0500)]
cephadm: populate ports if known and not included in unit.meta

This is only helpful for daemons deployed by older versions of cephadm
that have known ports based on their type.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: gather and report ports in 'orch ps' output
Sage Weil [Mon, 8 Mar 2021 19:14:49 +0000 (14:14 -0500)]
mgr/cephadm: gather and report ports in 'orch ps' output

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40122 into master
Patrick Donnelly [Mon, 15 Mar 2021 22:46:24 +0000 (15:46 -0700)]
Merge PR #40122 into master

* refs/pull/40122/head:
qa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10)

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agomgr/cephadm: fix 'auth caps' fallback 40140/head
Sage Weil [Mon, 15 Mar 2021 22:34:57 +0000 (17:34 -0500)]
mgr/cephadm: fix 'auth caps' fallback

The first get-or-create attempt also needs to tolerate failure.

Fixes: 8ceea1961f818dc2d07edf9c256ebe5150b6b133
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40078 into master
Sage Weil [Mon, 15 Mar 2021 20:11:24 +0000 (16:11 -0400)]
Merge PR #40078 into master

* refs/pull/40078/head:
mgr: wait for ~3 beacons on startup if mons are pre-pacific
mon/MgrMonitor: populate available_modules from promote_standby()

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #40071 into master
Sage Weil [Mon, 15 Mar 2021 20:10:34 +0000 (16:10 -0400)]
Merge PR #40071 into master

* refs/pull/40071/head:
mgr/cephadm: tolerate failure to update daemon caps
mgr/cephadm: fix get_keyring_with_caps

Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #40043 from cbodley/wip-49741
Casey Bodley [Mon, 15 Mar 2021 19:12:00 +0000 (15:12 -0400)]
Merge pull request #40043 from cbodley/wip-49741

rgw: rgw::sal::RGWBucket initializes creation_time

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39820 from cbodley/wip-qa-rgw-ignore-pg-avail
Casey Bodley [Mon, 15 Mar 2021 19:09:29 +0000 (15:09 -0400)]
Merge pull request #39820 from cbodley/wip-qa-rgw-ignore-pg-avail

qa/rgw: put PG_AVAILABILITY ignorelist override in its own file

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge PR #39979 into master
Sage Weil [Mon, 15 Mar 2021 18:49:24 +0000 (14:49 -0400)]
Merge PR #39979 into master

* refs/pull/39979/head:
python-common: fix PlacementSpec target size method
python-common: count-per-host must be combined with label or hosts or host_pattern
mgr/cephadm: handle bare 'count-per-host:NNN', fix comments
mgr/cephadm/schedule: remove Scheduler abstraction (for now at least)
mgr/cephadm/schedule: calculate additions/removals in place()
mgr/cephadm/schedule: allow colocation of certain daemon types
mgr/cephadm/schedule: shuffle candidates, not final placements
mgr/cephadm/schedule: pass per-type allow_colo to the scheduler
mgr/cephadm/services/cephadmservice: fix typo
mgr/cephadm/schedule: pass daemons, not get_daemons_func
mgr/cephadm: use local var
mgr/cephadm/schedule: move host filtering into get_candidates()
python-common/ceph/deployment/service_spec: disallow max-per-host + explicit placement
mgr/cephadm/schedule: respect count-per-host
mgr/cephadm: adjust deployment logic to allow multiple daemons per host
python-common: add count-per-host to PlacementSpec
mgr/cephadm: do not worry about even # of monitors

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoqa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10) 40122/head
Yuri Weinstein [Mon, 15 Mar 2021 16:52:42 +0000 (09:52 -0700)]
qa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agomgr/cephadm: less noise about refreshing hosts 40130/head
Sage Weil [Mon, 15 Mar 2021 18:47:39 +0000 (13:47 -0500)]
mgr/cephadm: less noise about refreshing hosts

These happen every ~10 minutes and will obscure any real messages of
interest.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40049 from gregsfortytwo/wip-stretch-fixes-2
Gregory Farnum [Mon, 15 Mar 2021 18:33:35 +0000 (11:33 -0700)]
Merge pull request #40049 from gregsfortytwo/wip-stretch-fixes-2

osd: Fix a bunch of stretch peering issues

Reviewed-by: Sam Just <sjust@redhat.com>
4 years agoMerge pull request #39858 from sseshasa/wip-dmclock-hdd-misc-imprvmts
Neha Ojha [Mon, 15 Mar 2021 17:27:56 +0000 (10:27 -0700)]
Merge pull request #39858 from sseshasa/wip-dmclock-hdd-misc-imprvmts

osd: Incorporate cost per io & cost per byte for cost scaling calculations in mclock scheduler.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoqa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus 40075/head
Sage Weil [Fri, 12 Mar 2021 16:49:10 +0000 (10:49 -0600)]
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus

centos currently has the semi-broken podman 2.2.1 (no json function
for inspect), but the latest octopus (15.2.10) works around it.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: tolerate failure to update daemon caps 40071/head
Sage Weil [Mon, 15 Mar 2021 16:55:36 +0000 (11:55 -0500)]
mgr/cephadm: tolerate failure to update daemon caps

If we're upgrading from 15.2.0, we may fail to update caps.  Instead of
failing the upgrade hard, warn to the log and continue.  This is less
than ideal, but the caps will get corrected the next time the daemon is
redeployed on the next upgrade, and most likely the previous caps will
continue to work (given they were presumably working before the upgrade).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agodoc: reorganize Windows docs 40114/head
Lucian Petrut [Mon, 15 Mar 2021 07:38:00 +0000 (07:38 +0000)]
doc: reorganize Windows docs

Most of the Windows documentation is currently included in the
README.windows.rst file.

To make it more accessible, we're moving most of it to the
"doc/" folder, adding the following pages:

* Installing Ceph on Windows
* RBD on Windows
* Windows troubleshooting

We'll keep the build and manual install instructions in
README.windows.rst. Note that ceph-dokan already has a separate
doc page.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agoMerge pull request #40098 from tchaikov/wip-qa-powertools
Kefu Chai [Mon, 15 Mar 2021 15:05:47 +0000 (23:05 +0800)]
Merge pull request #40098 from tchaikov/wip-qa-powertools

qa/workunits/rados/test_envlibrados_rocksdb: s/PowerTools/powertools/

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #40111 from tchaikov/wip-ceph-argparse-no-option-name
Kefu Chai [Mon, 15 Mar 2021 14:46:32 +0000 (22:46 +0800)]
Merge pull request #40111 from tchaikov/wip-ceph-argparse-no-option-name

pybind/ceph_argparse: do not print flag name before CephChoices in he…

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #39916 from zdover23/wip-doc-cephadm-adoption-preparation-2021...
Sebastian Wagner [Mon, 15 Mar 2021 14:25:39 +0000 (15:25 +0100)]
Merge pull request #39916 from zdover23/wip-doc-cephadm-adoption-preparation-2021-Mar-08

doc/cephadm: rewrite "preparation" in adoption.rst

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #40115 from tchaikov/wip-mgr-prometheus
Kefu Chai [Mon, 15 Mar 2021 14:17:24 +0000 (22:17 +0800)]
Merge pull request #40115 from tchaikov/wip-mgr-prometheus

pybind/mgr/prometheus: add typing annotations

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #40002 from orozery/rbd-encryption-api
Jason Dillaman [Mon, 15 Mar 2021 12:28:33 +0000 (08:28 -0400)]
Merge pull request #40002 from orozery/rbd-encryption-api

librbd: crypto format api semantics change

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agopybind/mgr/prometheus: add typing annotations 40115/head
Kefu Chai [Mon, 15 Mar 2021 11:35:16 +0000 (19:35 +0800)]
pybind/mgr/prometheus: add typing annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_argparse: do not print flag name before CephChoices in help descs 40111/head
Kefu Chai [Mon, 15 Mar 2021 10:27:44 +0000 (18:27 +0800)]
pybind/ceph_argparse: do not print flag name before CephChoices in help descs

This reverts commit 332abcd25059c3ed07ca5df54110a83e07f28f86.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40101 from dzafman/wip-49779
Kefu Chai [Mon, 15 Mar 2021 09:38:41 +0000 (17:38 +0800)]
Merge pull request #40101 from dzafman/wip-49779

standalone: osd-recovery-scrub.sh: Recovery never started

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #40056 from tchaikov/wip-no-more-MOSDPGNotify
Kefu Chai [Mon, 15 Mar 2021 09:21:39 +0000 (17:21 +0800)]
Merge pull request #40056 from tchaikov/wip-no-more-MOSDPGNotify

osd: do not send/handle MOSDPGNotify anymore

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36298 from p-se/mgr-prom-collect-time-metric
Kefu Chai [Mon, 15 Mar 2021 09:11:55 +0000 (17:11 +0800)]
Merge pull request #36298 from p-se/mgr-prom-collect-time-metric

mgr/prometheus: introduce metric for collection time

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40100 from liewegas/bug-49788
Kefu Chai [Mon, 15 Mar 2021 09:09:18 +0000 (17:09 +0800)]
Merge pull request #40100 from liewegas/bug-49788

osd: propagate base pool application_metadata to tiers

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40041 from adk3798/helpcoms
Kefu Chai [Mon, 15 Mar 2021 09:07:07 +0000 (17:07 +0800)]
Merge pull request #40041 from adk3798/helpcoms

pybind/ceph_argparse: print flag name before CephChoices in help descs

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40062 from tchaikov/wip-no-lockdep-in-rel-builds
Kefu Chai [Mon, 15 Mar 2021 09:03:00 +0000 (17:03 +0800)]
Merge pull request #40062 from tchaikov/wip-no-lockdep-in-rel-builds

cmake: do not build debug_mutex or lockdep for Release build

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agolibrbd: crypto format api semantics change 40002/head
Or Ozeri [Tue, 9 Mar 2021 20:14:49 +0000 (22:14 +0200)]
librbd: crypto format api semantics change

This commit alters the semantics of the encryption format api
to also load the encryption after format completes.
Additionally, several other small changes in librbd crypto are included,
in preparation of supporting clone formatting.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
4 years agotest: osd-recovery-scrub.sh: Test fails if no scrubs happened for a recovering pg 40101/head
David Zafman [Sat, 13 Mar 2021 05:56:28 +0000 (05:56 +0000)]
test: osd-recovery-scrub.sh: Test fails if no scrubs happened for a recovering pg

Change TEST_recovery_scrub_2 to create more objects and use
osd_recovery_sleep to prevent recovery from finihing before
we start to scrub.  Verify that at least 1 scrub was started
while the pg was reovering.

Fixes: https://tracker.ceph.com/issues/49779
Signed-off-by: David Zafman <dzafman@redhat.com>
4 years agocrimson/osd: do not include unused headers 40056/head
Kefu Chai [Fri, 12 Mar 2021 07:21:04 +0000 (15:21 +0800)]
crimson/osd: do not include unused headers

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd/PG: do not include unused headers
Kefu Chai [Fri, 12 Mar 2021 07:13:24 +0000 (15:13 +0800)]
osd/PG: do not include unused headers

neither MOSDPGNotify nor MOSDPGInfo is used in PG.cc, so drop the
corresponding headers.

the same appiles to some other types and headers declared and included
in PG.h

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd: do not handle pre-octopus messages
Kefu Chai [Fri, 12 Mar 2021 06:51:30 +0000 (14:51 +0800)]
osd: do not handle pre-octopus messages

MOSDPGQuery and MOSDPGInfo messages are sent by
pre-octopus OSD, so in quincy and up clusters, we do not need
to handle them anymore, as we can only upgrade from octopus and
up to quincy.

we can drop MOSDPGNotify after Q + 2, though, after we stop sending
MOSDPGNotify in Q release.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd: send MOSDPGNotify2 instead of MOSDPGNotify
Kefu Chai [Fri, 12 Mar 2021 07:09:12 +0000 (15:09 +0800)]
osd: send MOSDPGNotify2 instead of MOSDPGNotify

as we prefer sending MOSDPGNotify2 over MOSDPGNotify in PeeringState
in post octopus, to be more consistent and have one less thing to
worry, let's just use MOSDPGNotify2 in OSD.cc as well.

Fixes: https://tracker.ceph.com/issues/49758
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd: drop OSD::create_context()
Kefu Chai [Fri, 12 Mar 2021 06:40:47 +0000 (14:40 +0800)]
osd: drop OSD::create_context()

OSD::create_context() was used for creating PeeringCtx from OSD's
require_osd_release. but since the check against require_osd_release
is not required anymore, let's drop this helper.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd/PeeringState: do not check for require_osd_release
Kefu Chai [Fri, 12 Mar 2021 06:23:50 +0000 (14:23 +0800)]
osd/PeeringState: do not check for require_osd_release

before this change, we always check for require_osd_release when
creating MOSDPGNotify2 or MOSDPGNotify, if require_osd_release is
greater or equal to octopus, MOSDPGNotify2 is created.

since we are in a post-quincy era, and we only need to upgrade from
octopus and up to quincy, there is no need to be compatible with
osd whose version is lower than octopus.

in this change, the check in `BufferedRecoveryMessages::send_notify()`
is dropped.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoRevert "qa/standalone/scrub/osd-recovery-scrub: fix unnoticed recovery state"
David Zafman [Sat, 13 Mar 2021 05:32:41 +0000 (05:32 +0000)]
Revert "qa/standalone/scrub/osd-recovery-scrub: fix unnoticed recovery state"

This reverts commit 1323bdb839e77fb27cba36ef2725bb7f163b1db4.

The tests needs to scrub while recovery is in progress, so catching
recovery from the logs after the fact isn't the proper setup.
We can use osd_recovery_sleep config.

Signed-off-by: David Zafman <dzafman@redhat.com>
4 years agoosd: propagate base pool application_metadata to tiers 40100/head
Sage Weil [Sat, 13 Mar 2021 16:34:43 +0000 (11:34 -0500)]
osd: propagate base pool application_metadata to tiers

If there is application metadata on the base pool, it should be mirrored
to any other tiers in the set.  This aligns with the fact that the
'ceph osd pool application ...' commands refuse to operate on a non-base
pool.

This fixes problems with accessing tiers (e.g., cache tiers) when the
cephx cap is written in terms of application metadata.

Fixes: https://tracker.ceph.com/issues/49788
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40042 into master
Sage Weil [Sat, 13 Mar 2021 15:40:55 +0000 (10:40 -0500)]
Merge PR #40042 into master

* refs/pull/40042/head:
qa/suites/rados/cephadm: add back centos+rhel with kubic podman

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoqa/workunits/rados/test_envlibrados_rocksdb: s/PowerTools/powertools/ 40098/head
Kefu Chai [Sat, 13 Mar 2021 09:00:56 +0000 (17:00 +0800)]
qa/workunits/rados/test_envlibrados_rocksdb: s/PowerTools/powertools/

see also 7a68845579c701f43c9526c810e586680cf76fe0

Fixes: https://tracker.ceph.com/issues/49787
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not build lockdep for Release build 40062/head
Kefu Chai [Fri, 12 Mar 2021 11:39:28 +0000 (19:39 +0800)]
cmake: do not build lockdep for Release build

lockdep create large data structures on .bss and on heap for tracking
the locks and their dependencies. but we don't need to pay for this
if lockdep is not enabled.

lockdep helps us to track the lock dependencies related issue on Debug
build. and Release build, this feature hurts the performance and more
importantly, lockdeps is a feature only kicks in when using the
mutex_debug and friends. they are not used in Release build at all.

so, after this change, lockdep is not built in Release build. and
the static variables defined in lockdep.cc are not allocated anymore
in Release build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/lockdep: disable lockdep if !CEPH_DEBUG_MUTEX
Kefu Chai [Fri, 12 Mar 2021 13:12:08 +0000 (21:12 +0800)]
common/lockdep: disable lockdep if !CEPH_DEBUG_MUTEX

make lockdep a feature which can only be enabled at compile time.

we don't need to have lockdep when CEPH_DEBUG_MUTEX is OFF, so
just comment it out in Release builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40036 from tchaikov/wip-doc-mgr
Josh Durgin [Fri, 12 Mar 2021 23:48:10 +0000 (15:48 -0800)]
Merge pull request #40036 from tchaikov/wip-doc-mgr

doc/mgr: fix formatting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #40044 from ceph/wip-1427-notes
Josh Durgin [Fri, 12 Mar 2021 23:35:19 +0000 (15:35 -0800)]
Merge pull request #40044 from ceph/wip-1427-notes

doc: 14.2.17 Release Notes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
4 years agoosd: PeeringState: implement an acting_set_writeable() function 40049/head
Greg Farnum [Fri, 12 Mar 2021 22:41:03 +0000 (22:41 +0000)]
osd: PeeringState: implement an acting_set_writeable() function

Use it instead of direct checks against min_size and stretch_set_can_peer()
when deciding whether to go STATE_ACTIVE/STATE_PEERED or do updates
to things like last_epoch_started.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: fix a boolean conditional direction
Greg Farnum [Fri, 12 Mar 2021 21:05:29 +0000 (21:05 +0000)]
osd: PeeringState: fix a boolean conditional direction

We want to add an OSD from the mandatory member if we DON'T already have one!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: fix stretch peering so PGs can go peered but not active
Greg Farnum [Fri, 12 Mar 2021 20:13:38 +0000 (20:13 +0000)]
osd: PeeringState: fix stretch peering so PGs can go peered but not active

I misunderstood and there was a pretty serious error here: to prevent
accidents, choose_acting() was preventing PGs from *finishing* peering
if they didn't satisfy the stretch cluster rules. What we actually want
to do is to finish peering, but not go active.

Happily, this is easy to fix -- we just add a call to stretch_set_can_peer()
alongside existing min_size checks when we choose whether to go PG_STATE_ACTIVE
or PG_STATE_PEERED!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: don't add acting-set OSDs to candidate set in stretch mode
Greg Farnum [Thu, 11 Mar 2021 22:19:10 +0000 (22:19 +0000)]
osd: PeeringState: don't add acting-set OSDs to candidate set in stretch mode

We were adding them once from the acting set, and then once from the all_infos
set, and that hit an assert later on. (I think it was otherwise harmless, but
I don't want to weaken the assert!)

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: fix calc_replicated_acting_stretch() syntax/logic
Greg Farnum [Thu, 11 Mar 2021 10:26:47 +0000 (10:26 +0000)]
osd: PeeringState: fix calc_replicated_acting_stretch() syntax/logic

There was a major error here! get_ancestor() was type-deduced to return
a bucket_candidates_t -- a *copy* of what was in the map, not the reference
to it we wanted to actually amend!

Fix this by returning a pointer instead. There's a way to coerce things
to return a reference instead but the syntax seems clumsier to me
and I'm not familiar with it anyway -- this works just fine.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: respect stretch peering constraints for async recovery
Greg Farnum [Thu, 11 Mar 2021 07:40:52 +0000 (07:40 +0000)]
osd: PeeringState: respect stretch peering constraints for async recovery

Happily this is pretty simple: we just need to check that the resulting
wanted set can peer, which we have a function for. Run it before actually
swapping the want and candidate_want sets.

If we're not in stretch mode, this is a cheap function call that
will always return true, so it's equivalent to what we already have for them.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agoosd: PeeringState: add a comment about using size as a proxy for activateable
Greg Farnum [Thu, 11 Mar 2021 07:19:26 +0000 (07:19 +0000)]
osd: PeeringState: add a comment about using size as a proxy for activateable

When reviewing, I mistakenly thought we needed to skip a size check in
choose_acting() in case of mismatches between size and bucket counts, but that
is not accurate!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
4 years agomgr: wait for ~3 beacons on startup if mons are pre-pacific 40078/head
Sage Weil [Fri, 12 Mar 2021 20:21:49 +0000 (15:21 -0500)]
mgr: wait for ~3 beacons on startup if mons are pre-pacific

If we are going active and the mons are pre-pacific, they may have the
bug https://tracker.ceph.com/issues/49778 which prevents our modules
metadata (including options) from being updated (until the next beacon).
Wait a bit (6s by default, 3x the 2s mgr_tick_period) to let this
happen.

This allows us to upgrade from broken pre-pacific mons using cephadm,
which may (if orig cluster is <15.2.5) immediately do a cephadm
migration that relies on the mgr/cephadm/migration_current config
option being present in the mon's mgrmap.

Workaround for https://tracker.ceph.com/issues/49778

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40045 into master
Sage Weil [Fri, 12 Mar 2021 20:09:15 +0000 (15:09 -0500)]
Merge PR #40045 into master

* refs/pull/40045/head:
cephadm: use image id, not name, when inspecting for RepoDigests

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agomon/MgrMonitor: populate available_modules from promote_standby()
Sage Weil [Fri, 12 Mar 2021 20:00:49 +0000 (15:00 -0500)]
mon/MgrMonitor: populate available_modules from promote_standby()

This was done in the beacon path, where there is no active mgr and we
get a new entrant, but not for this case where an existing standby is
promoted to active.

This fixes a problem during upgrade where a new (standby) mgr's modules
have a new module option but it is not reflected immediately (not until
the next beacon).

Fixes: https://tracker.ceph.com/issues/49778
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix get_keyring_with_caps
Sage Weil [Fri, 12 Mar 2021 16:15:35 +0000 (10:15 -0600)]
mgr/cephadm: fix get_keyring_with_caps

1- Pass caps to 'auth get-or-create'
2- Only try 'auth caps' if the get-or-create failed

Note that the 'auth caps' step can fail if upgrading from 15.2.0 since
'profile mgr' didn't include 'auth caps' until 15.2.1.  We're not
addressing that for now...

Fixes: 7c0d532f3a4839f4199a13773fb5fa8b6fb3f183
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40052 from tchaikov/wip-rpm-no-rbd-rwl-on-el8
Kefu Chai [Fri, 12 Mar 2021 17:56:19 +0000 (01:56 +0800)]
Merge pull request #40052 from tchaikov/wip-rpm-no-rbd-rwl-on-el8

ceph.spec: build with system libpmem on fedora and el8

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #40050 from dillaman/wip-49690
Mykola Golub [Fri, 12 Mar 2021 17:50:04 +0000 (19:50 +0200)]
Merge pull request #40050 from dillaman/wip-49690

librbd/io: send alloc_hint when compression hint is set

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #38853 from dzafman/wip-48843
David Zafman [Fri, 12 Mar 2021 17:20:51 +0000 (09:20 -0800)]
Merge pull request #38853 from dzafman/wip-48843

osd: Try other PGs when reservation failures occur

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agocommon/ceph_context: disable lockdep if !CEPH_DEBUG_MUTEX
Kefu Chai [Fri, 12 Mar 2021 11:50:45 +0000 (19:50 +0800)]
common/ceph_context: disable lockdep if !CEPH_DEBUG_MUTEX

make lockdep a feature which can only be enabled at compile time.

we don't need to have lockdep when CEPH_DEBUG_MUTEX is OFF, so
just comment it out in Release builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not build mutex_debug.cc if !WITH_CEPH_DEBUG_MUTEX
Kefu Chai [Fri, 12 Mar 2021 11:32:16 +0000 (19:32 +0800)]
cmake: do not build mutex_debug.cc if !WITH_CEPH_DEBUG_MUTEX

there is no need to build shared_mutex_debug.cc and
mutex_debug.cc, if they are not used at all. in Release build
we just use the mutex primitives offered by C++ standard library and
the POSIX API offered by libc.

Signed-off-by: Kefu Chai <kchai@redhat.com>