]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agorgw: allow rgw-orphan-list to handle intermediate files w/ binary data 39764/head
J. Eric Ivancich [Thu, 4 Feb 2021 21:19:17 +0000 (16:19 -0500)]
rgw: allow rgw-orphan-list to handle intermediate files w/ binary data

Updates the grep commands with the --text command-line option since
there are cases where rados object names contain NUL characters.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit b520cc5a910ea4fba0d2023b13e22b4166f767c7)

4 years agoMerge pull request #39188 from liewegas/pr-39173-pacific
Yuri Weinstein [Mon, 1 Mar 2021 18:19:59 +0000 (10:19 -0800)]
Merge pull request #39188 from liewegas/pr-39173-pacific

pacific: mgr/devicehealth: do not create pool until there are enough osds

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39189 from liewegas/pr-39172-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:37:56 +0000 (09:37 -0800)]
Merge pull request #39189 from liewegas/pr-39172-pacific

pacific: mgr/orchestrator: remove 'host' arg for 'orch ls'

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39236 from k0ste/wip-49108-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:36:25 +0000 (09:36 -0800)]
Merge pull request #39236 from k0ste/wip-49108-pacific

pacific: mgr/zabbix: format ceph.[{#POOL},percent_used as float

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39663 from pponnuvel/wip-49442-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:21:12 +0000 (09:21 -0800)]
Merge pull request #39663 from pponnuvel/wip-49442-pacific

pacific: rgw: radosgw-admin: clarify error when email address already in use

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #39048 from smithfarm/wip-48971-pacific
Yuri Weinstein [Mon, 1 Mar 2021 17:20:33 +0000 (09:20 -0800)]
Merge pull request #39048 from smithfarm/wip-48971-pacific

pacific: rgw: Initialize Layout Logs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge PR #39483 into pacific
Sage Weil [Sun, 28 Feb 2021 16:07:12 +0000 (10:07 -0600)]
Merge PR #39483 into pacific

* refs/pull/39483/head:
mgr/cephadm: based sleep interval on configured intervals/timeouts

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge PR #39595 into pacific
Sage Weil [Sun, 28 Feb 2021 16:06:56 +0000 (10:06 -0600)]
Merge PR #39595 into pacific

* refs/pull/39595/head:
mon/KVMonitor: fix 'osd new' cross-service commit
mgr/ActivePyModules.cc: use wrappers for acquiring/releasing GIL
mgr: rename config_prefix -> mgr_store_prefix
mgr: fix config option prefix
mgr/MgrStandby: simplify weird code
mgr_module: get_foreign_ceph_option(entity, name)
mgr: maintain a ConfigMap copy
mon/ConfigMap: factor out parse_key helper
mon/ConfigMap: fix stray option leak
mgr: use new kv subscription for mgr/, device/, config/
mon: allow subscription to kv/config-key data
mon: convert ConfigKeyService -> KVMonitor

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agomon/KVMonitor: fix 'osd new' cross-service commit 39595/head
Sage Weil [Wed, 24 Feb 2021 21:20:18 +0000 (16:20 -0500)]
mon/KVMonitor: fix 'osd new' cross-service commit

When we converted ConfigKeyService to KVMonitor, we didn't correctly
change this to propose_pending(), which mean that the kv change wasn't
captured in the paxos transaction.

Fixes: bb7ebc41532aeb23cff2241ab07b3f01c2f57ddd
Fixes: https://tracker.ceph.com/issues/49460
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 66891b4845fbf119cacb2c77d39180e28c6626d5)

4 years agoMerge pull request #39401 from cbodley/wip-48948
Casey Bodley [Thu, 25 Feb 2021 16:06:07 +0000 (11:06 -0500)]
Merge pull request #39401 from cbodley/wip-48948

pacific: rgw: avoid (rgw_file) illegal access to stack memory

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39443 from cbodley/wip-qa-rgw-pacific
Casey Bodley [Thu, 25 Feb 2021 16:05:58 +0000 (11:05 -0500)]
Merge pull request #39443 from cbodley/wip-qa-rgw-pacific

pacific: qa/rgw: fixes for keystone, vault, barbican tests

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #39444 from cbodley/wip-49060
Casey Bodley [Thu, 25 Feb 2021 16:05:47 +0000 (11:05 -0500)]
Merge pull request #39444 from cbodley/wip-49060

pacific: rgw: fix valgrind errors when protected_fixedsize_stack is used

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #39445 from cbodley/wip-49129
Casey Bodley [Thu, 25 Feb 2021 16:05:36 +0000 (11:05 -0500)]
Merge pull request #39445 from cbodley/wip-49129

pacific: rgw: fix trailing null in object names of multipart reuploads

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39446 from cbodley/wip-49172
Casey Bodley [Thu, 25 Feb 2021 16:05:28 +0000 (11:05 -0500)]
Merge pull request #39446 from cbodley/wip-49172

pacific: rgw multisite: fix RGWCoroutine error handling

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39447 from cbodley/wip-49251
Casey Bodley [Thu, 25 Feb 2021 16:04:35 +0000 (11:04 -0500)]
Merge pull request #39447 from cbodley/wip-49251

pacific: rgw: fix snprintf() truncate warning in ETagVerifier

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39448 from cbodley/wip-49030
Casey Bodley [Thu, 25 Feb 2021 16:04:12 +0000 (11:04 -0500)]
Merge pull request #39448 from cbodley/wip-49030

pacific: cmake: boost>=1.74 adds BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT to radosgw

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39449 from cbodley/wip-49071
Casey Bodley [Thu, 25 Feb 2021 16:04:01 +0000 (11:04 -0500)]
Merge pull request #39449 from cbodley/wip-49071

pacific: rgw : add MD5 in forward_request

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39553 from cbodley/wip-49358
Casey Bodley [Thu, 25 Feb 2021 16:03:36 +0000 (11:03 -0500)]
Merge pull request #39553 from cbodley/wip-49358

pacific: rgw: fix buffer overflow in s3 select op

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #39623 from sebastian-philipp/pacific-backport-39259-39238-39261...
Sebastian Wagner [Thu, 25 Feb 2021 11:11:43 +0000 (12:11 +0100)]
Merge pull request #39623 from sebastian-philipp/pacific-backport-39259-39238-39261-38897-39193-38695-39266-39350-39153-37729-

pacific: cephadm Batch backport Febrary (2)

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #39636 from ceph/backport_39398_39424_39490
Sebastian Wagner [Thu, 25 Feb 2021 09:59:40 +0000 (10:59 +0100)]
Merge pull request #39636 from ceph/backport_39398_39424_39490

pacific: cephadm: Fix selinux

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39557 from liewegas/pr-39304-pacific
Yuri Weinstein [Thu, 25 Feb 2021 00:01:39 +0000 (16:01 -0800)]
Merge pull request #39557 from liewegas/pr-39304-pacific

pacific: qa/suites/upgrade/octopus-x: fix rgw ragweed test

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #39641 from dillaman/wip-librbd-backports-pacific-5
Jason Dillaman [Wed, 24 Feb 2021 18:04:59 +0000 (13:04 -0500)]
Merge pull request #39641 from dillaman/wip-librbd-backports-pacific-5

pacific: rbd: miscellaneous backports

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agotest/librbd: TestLibRBD.RenameViaLockOwner doesn't require journaling now 39641/head
Mykola Golub [Tue, 23 Feb 2021 04:46:00 +0000 (04:46 +0000)]
test/librbd: TestLibRBD.RenameViaLockOwner doesn't require journaling now

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 3042eec0d8aa7e28a8b0452c11a6dc6528fe4992)

4 years agolibrbd: use on-disk image name when storing mirror snapshot state
Mykola Golub [Mon, 22 Feb 2021 12:54:43 +0000 (12:54 +0000)]
librbd: use on-disk image name when storing mirror snapshot state

Fixes: https://tracker.ceph.com/issues/49115
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 813728ae2036231a269be86bcb3dc48951a20f21)

4 years agotest/librbd: extend TestLibRBD.RenameViaLockOwner
Mykola Golub [Mon, 22 Feb 2021 12:53:38 +0000 (12:53 +0000)]
test/librbd: extend TestLibRBD.RenameViaLockOwner

To cover the following case:

- Client A has image opened but does not owns the lock.
- Client B renames the image (client A is not aware of it).
- Client A becomes the lock owner.
- Client B requests rename, which is proxied to the client A.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 2d2e04e86bcc666e3abaceeeef7e1598dcc9fb94)

4 years agolibrbd: make rename be always executed by lock owner
Mykola Golub [Sun, 14 Feb 2021 11:50:32 +0000 (11:50 +0000)]
librbd: make rename be always executed by lock owner

Fixes: https://tracker.ceph.com/issues/49115
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 2a9fac2cc4b10af04c52e12f34932e6d2d91441f)

4 years agolibrbd: always check on-disk image name when renaming
Mykola Golub [Sun, 14 Feb 2021 11:48:35 +0000 (11:48 +0000)]
librbd: always check on-disk image name when renaming

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 950a7b5cfc66bdfea35304e6cb06e74c92dcaf5c)

4 years agorbd-mirror: reset update_status_task pointer in timer thread
Mykola Golub [Mon, 22 Feb 2021 16:22:54 +0000 (16:22 +0000)]
rbd-mirror: reset update_status_task pointer in timer thread

To avoid a time window when m_update_status_task is invalid. If
during this time the cancel_update_mirror_image_replay_status is
called, it may cancel some other's ImageReplayer task, if it
happened to add the task with the same address.

Fixes: https://tracker.ceph.com/issues/49418
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 3a289d43f6ab010bcadb80888fb73763f4f55ed0)

4 years agoMerge pull request #39642 from idryomov/wip-rbd-map-sanity-check-pacific
Ilya Dryomov [Wed, 24 Feb 2021 13:03:49 +0000 (14:03 +0100)]
Merge pull request #39642 from idryomov/wip-rbd-map-sanity-check-pacific

pacific: krbd: make sure the device node is accessible after the mapping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agorgw/radosgw-admin clarify error when email address already in use 39663/head
Matthew Vernon [Thu, 4 Feb 2021 11:41:14 +0000 (11:41 +0000)]
rgw/radosgw-admin clarify error when email address already in use

The error message if you try and create an S3 user with an email
address that is already associated with another S3 account is very
confusing; this patch makes it much clearer

To reproduce:

radosgw-admin user create --uid=foo --display-name="Foo test" --email=bar@domain.invalid
radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to parse parameters, user id mismatch, operation id: foo does not match: test

With this patch:

radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to create user test because user id foo already exists with email bar@domain.invalid

Fixes: https://tracker.ceph.com/issues/49137
Fixes: https://tracker.ceph.com/issues/19411
Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 05318d6f71e45a42a46518a0ef17047dfab83990)

4 years agoMerge pull request #39480 from kotreshhr/ceph-vol-client-update-fix-volumes-key-pacific
Ramana Raja [Wed, 24 Feb 2021 03:58:13 +0000 (22:58 -0500)]
Merge pull request #39480 from kotreshhr/ceph-vol-client-update-fix-volumes-key-pacific

pacific: pybind/ceph_volume_client: Update the 'volumes' key to 'subvolumes' in auth-metadata file

Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoMerge pull request #39637 from kamoltat/wip-ksirivad-pacific-backports-39422
Josh Durgin [Tue, 23 Feb 2021 19:27:12 +0000 (11:27 -0800)]
Merge pull request #39637 from kamoltat/wip-ksirivad-pacific-backports-39422

pacific: PendingReleaseNotes: mgr/pg_autoscaler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agokrbd: make sure the device node is accessible after the mapping 39642/head
Ilya Dryomov [Fri, 19 Feb 2021 15:47:17 +0000 (16:47 +0100)]
krbd: make sure the device node is accessible after the mapping

We have always assumed this to be the case and users' scripts and
orchestration tools have grown to depend on this.  Let's add some
enforcement, prompted by [1]:

  "I am running my Kubernetes worker node inside of an LXC container
   which doesn't benefit from the device node created by the kernel, so
   I'm using udev to create the /dev/rbd* device nodes inside of the LXC
   container."

which, through the unfortunate interaction with ceph-csi rbd plugin,
results in data loss for "volumeMode: Filesystem" PVs because it ends
up recreating the filesystem every time the PV is attached to the pod:

  "When deleting the pod and re-creating it, I can see that the RBD
   image is indeed being reformatted. This seems to be because when
   blkid is being run to check if the image is formatted, the /dev/rbd*
   device has not yet been created by udev. By the time the code gets
   down to running mkfs, the device is there and the damage is done."

[1] https://github.com/ceph/ceph-csi/issues/1820

Fixes: https://tracker.ceph.com/issues/49410
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f6854ac65d2a838e2f523979e341136e5f201b5c)

4 years agomgr/volumes: Bump up AuthMetadataManager's version 39480/head
Kotresh HR [Fri, 19 Feb 2021 11:27:23 +0000 (16:57 +0530)]
mgr/volumes: Bump up AuthMetadataManager's version

With ceph_volume_client and mgr-volumes co-existing
for sometime, the version of both needs to be same.
The ceph_volume_client version <=5 can't decode
'subvolumes' key in auth-metadata file. Hence to
handle version in-compatibility, the version of
ceph_volume_client is bumped up to 6 and the same
needs to be done in mgr-volume's AuthMetadataManager

Fixes: https://tracker.ceph.com/issues/49374
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 818c7781ff0467c6521bf2b3002b094fb8a71257)

4 years agopybind/ceph_volume_client: Bump up the version and compat_version to 6
Kotresh HR [Fri, 19 Feb 2021 11:12:33 +0000 (16:42 +0530)]
pybind/ceph_volume_client: Bump up the version and compat_version to 6

With 'volumes' key updated to 'subvolumes', the version of
ceph_volume_client <= 5 can't decode auth-metadata file. Hence
bumping up ceph_volume_client version and compat_version to 6.

Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
4 years agorbd: fix rbd-wnbd log messages
Lucian Petrut [Tue, 23 Feb 2021 12:37:59 +0000 (12:37 +0000)]
rbd: fix rbd-wnbd log messages

Previously, rbd-wnbd was called "rbd-nbd" as it was using NBD
to communicate with the driver.

The tool was then renamed to "rbd-wnbd" but some messages were
not updated.

This patch will simply replace "rbd-nbd" with "rbd-wnbd" in some
log messages.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 108f4363ae31e94928792e7ab3ae1a2cbd5e3c41)

4 years agorbd: fix rbd-wnbd device status
Lucian Petrut [Wed, 17 Feb 2021 13:27:11 +0000 (13:27 +0000)]
rbd: fix rbd-wnbd device status

The "rbd-wnbd show" command will always report the device status
as "inactive". This patch adds the missing check, similar to the
one used by the "list" command.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 94cf22e35217e33a48ab73a018c8b1015b887ca4)

4 years agocommon: fix win32 event log source
Lucian Petrut [Wed, 17 Feb 2021 12:49:02 +0000 (12:49 +0000)]
common: fix win32 event log source

The Windows "get_process_name" function uses the input buffer
to store the entire executable path, while the caller only
expects the filename.

The "get_process_name_cpp" function is using an insufficient
buffer, for which reason it will return "(unknown)" when the
executable path exceeds 32 characters.

Windows event log entries have the wrong source because of this.

We'll update "get_process_name" to use a separate buffer for the
full executable path and avoid requesting a larger buffer than
actually needed.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 86c7c2f5be96e80d492447d6582131f1b2d3438d)

4 years agodoc/orch: Host labels 39623/head
Sebastian Wagner [Mon, 15 Feb 2021 12:34:47 +0000 (13:34 +0100)]
doc/orch: Host labels

Fixes: https://tracker.ceph.com/issues/45833
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 4cb501416da0f89648f4f075b447a35f45feed7f)

4 years agodoc/cephadm: Disable automatic deployment of daemons
Sebastian Wagner [Mon, 15 Feb 2021 10:39:33 +0000 (11:39 +0100)]
doc/cephadm: Disable automatic deployment of daemons

Fixes: https://tracker.ceph.com/issues/45767
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5c872ace45d2a2f4cb6e42802c1a658fe334461b)

4 years agodoc/cephadm: Per-service and per-daemon events
Sebastian Wagner [Mon, 15 Feb 2021 13:20:13 +0000 (14:20 +0100)]
doc/cephadm: Per-service and per-daemon events

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a9b947b51e773e09e05feb7fa718ad247ec4ec8c)

4 years agodoc/cephadm: Restoring the MON quorum
Sebastian Wagner [Mon, 15 Feb 2021 12:01:01 +0000 (13:01 +0100)]
doc/cephadm: Restoring the MON quorum

Fixes: https://tracker.ceph.com/issues/49214
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 02584d3ee772e4d5d8d7b4858dff192cd49c5281)

4 years agomgr/cephadm: _add_daemon: remove config_func
Sebastian Wagner [Fri, 5 Feb 2021 13:14:25 +0000 (14:14 +0100)]
mgr/cephadm: _add_daemon: remove config_func

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1e3d98ee1e15744c1eb57bba578765bbaeb1096d)

4 years agomgr/cephadm: Add config() to CephadmService
Sebastian Wagner [Mon, 1 Feb 2021 17:13:59 +0000 (18:13 +0100)]
mgr/cephadm: Add config() to CephadmService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1b4e9e68d449ba58f1d0169c733124879173cf76)

4 years agomgr/orch: remove a bunch of duplicated code
Sebastian Wagner [Sat, 6 Feb 2021 00:05:03 +0000 (01:05 +0100)]
mgr/orch: remove a bunch of duplicated code

`ceph daemon add [..]` are all the same thing

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7f51e3ae975c7c84281696051d4e10e5a88316c2)

4 years agomgr/orch: remove a bunch of duplicated code
Sebastian Wagner [Fri, 5 Feb 2021 23:57:24 +0000 (00:57 +0100)]
mgr/orch: remove a bunch of duplicated code

We only need to call self.apply()

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5a49f253e11d7a907ebc48202173f60da42f3c9c)

4 years agomgr/cephadm: SpecStore: use d-under accessor methods
Sebastian Wagner [Sat, 13 Feb 2021 12:56:34 +0000 (13:56 +0100)]
mgr/cephadm: SpecStore: use d-under accessor methods

This fixes a conflict between:

1fcdad17685a8884135a2cda0cdfe3525e325474
2c7f899124207d0d4866e16d9eb5609777ea22e8

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b126628460ff46fc802e648c570a001695e30042)

4 years agocephadm: allow redeploy of daemons in error state if container running
Adam King [Mon, 8 Feb 2021 19:55:45 +0000 (14:55 -0500)]
cephadm: allow redeploy of daemons in error state if container running

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 71613e61fa66e3479693f61ce63732550f756e17)

4 years agomgr/cephadm: make generate_config internal
Adam King [Mon, 8 Feb 2021 13:43:59 +0000 (08:43 -0500)]
mgr/cephadm: make generate_config internal

makes generate_config into an internal function for prepare_create

Fixes: https://tracker.ceph.com/issues/48373
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2c7f899124207d0d4866e16d9eb5609777ea22e8)

4 years agomgr/cephadm: call prepare_create before redeploy
Adam King [Thu, 28 Jan 2021 18:32:01 +0000 (13:32 -0500)]
mgr/cephadm: call prepare_create before redeploy

some services (notably iscsi and ha-rgw currently) rely
on actions in prepare_create to setup their configuration
even when being redeployed. This change makes sure prepare_create
is called before redeploys and reconfigurations the same way it is
done during initial deployment to avoid disrepencies. OSDs are
skipped as they do not have a normal prepare_create

Fixes: https://tracker.ceph.com/issues/48373
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 6fc6db23bf72d5d488fa983a7fd2491c5a9805d3)

4 years agomgr/cephadm: Put together default container images references
Juan Miguel Olmo Martínez [Fri, 12 Feb 2021 13:09:17 +0000 (14:09 +0100)]
mgr/cephadm: Put together default container images references

Placed all in the same location in order to make easy downstream modifications
and future changes

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit ce246479443a64b292c7cff2a662161c8a598e09)

Conflicts:
src/cephadm/cephadm

4 years agocephadm: --orphan-initial-daemons now applies services unmanaged
Sebastian Wagner [Fri, 5 Feb 2021 12:19:21 +0000 (13:19 +0100)]
cephadm: --orphan-initial-daemons now applies services unmanaged

We do not want to have orphaned daemons at all. Thus apply
services as unmanaged in order to run `ceph orch daemon add ...`
later.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit db5d3d4bd38d5f6fcbb95538c024899174cbc54f)

4 years agomgr/cephadm: purge service: remove `mds_join_fs`
Sebastian Wagner [Fri, 29 Jan 2021 17:49:25 +0000 (18:49 +0100)]
mgr/cephadm: purge service: remove `mds_join_fs`

Also add a unit test

Fixes: https://tracker.ceph.com/issues/48597
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 30d03ee07c75da87496eae34e2186b5531b4101c)

4 years agomgr/cephadm: Add service deletion timestamp to `orch ls`
Sebastian Wagner [Fri, 29 Jan 2021 17:27:52 +0000 (18:27 +0100)]
mgr/cephadm: Add service deletion timestamp to `orch ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 84ac0d3de167e5edfc36912420da102bd74279c6)

4 years agomgr/cephadm: mandate the existence of Services for Daemons
Sebastian Wagner [Fri, 29 Jan 2021 16:18:50 +0000 (17:18 +0100)]
mgr/cephadm: mandate the existence of Services for Daemons

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 745aa00909fe072f127ede7a5da77a489dc067ae)

4 years agomgr/cephadm: Add pytest to verify delayed service deletion
Sebastian Wagner [Fri, 29 Jan 2021 16:16:55 +0000 (17:16 +0100)]
mgr/cephadm: Add pytest to verify delayed service deletion

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 62267d2d3ca6a7626479947c061dcb0d168c12e5)

4 years agomgr/cephadm: Purge deleted services
Sebastian Wagner [Wed, 13 Jan 2021 13:12:24 +0000 (14:12 +0100)]
mgr/cephadm: Purge deleted services

If no daemon is deployed for a deleted service,
purge it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f73fdee1b081928051a5f52d510458f3b81525bd)

4 years agomgr/cephadm: add spec_deleted datetime to spec store
Sebastian Wagner [Wed, 13 Jan 2021 12:40:16 +0000 (13:40 +0100)]
mgr/cephadm: add spec_deleted datetime to spec store

We'd like to keep the spec in the store, while we're cleaning
up all the daemons. Reason is to clean up the
config and config-key store when the last daemon of a
service is deleted.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1fcdad17685a8884135a2cda0cdfe3525e325474)

4 years agomgr/cephadm: remove dead code
Sebastian Wagner [Wed, 13 Jan 2021 12:43:21 +0000 (13:43 +0100)]
mgr/cephadm: remove dead code

SpecStore.find is no longer used

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 9869b61e309034544b9e2ea92610840bdf34a875)

4 years agomgr/cephadm: enable use_repo_digest by default for pacific
Sebastian Wagner [Mon, 25 Jan 2021 10:54:49 +0000 (11:54 +0100)]
mgr/cephadm: enable use_repo_digest by default for pacific

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 54ac36e15d899c84324fb93b46ef99fd85cd91dd)

4 years agocephadm: add error messages to log file
Michael Fritch [Tue, 9 Feb 2021 18:12:37 +0000 (11:12 -0700)]
cephadm: add error messages to log file

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3e503d148749fb78ccbf4adc1ecdfff6b8524df2)

4 years agoqa/suites/rados/cephadm/upgrade: change starting version by distro
Sage Weil [Thu, 11 Feb 2021 19:08:19 +0000 (14:08 -0500)]
qa/suites/rados/cephadm/upgrade: change starting version by distro

centos/rhel have podman 2, which does not like conflicting --cap-add and
--privileged arguments.  cephadm versions prior to 15.2.5 use both args,
however, which means the rhel/centos upgrade test has to start at 15.2.5
to work at all on those distros (with the updated podman).

Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 38e14f923508efc4a18e942869065771598ef915)

4 years agodoc/dev/cephadm: scalability notes
Sebastian Wagner [Wed, 9 Dec 2020 10:59:59 +0000 (11:59 +0100)]
doc/dev/cephadm: scalability notes

Add a few notes about possible future scalability improvements

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f82e37434188550d9aadccab872d1356997cf661)

Conflicts:
doc/dev/cephadm/index.rst

4 years agomgr/orch: Fix flake8 error
Sebastian Wagner [Mon, 8 Feb 2021 16:37:52 +0000 (17:37 +0100)]
mgr/orch: Fix flake8 error

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit be9b98a4a38bb9aebaf010247d7915f77218d912)

4 years agocephadm: configure journald as the logdriver
Michael Fritch [Fri, 5 Feb 2021 01:19:25 +0000 (18:19 -0700)]
cephadm: configure journald as the logdriver

Introduced by e6792f306ab4d07251588fdca6ed3876ae3a092a

podman containers running in a detached `-d` state need to have a
configured logdriver for `cephadm logs` to be able to gather the daemon
output from journald

Fixes: https://tracker.ceph.com/issues/47916
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7c44f21bbbb0426fc2f83cfff7966c8bb9a71641)

4 years agomgr/cephadm: Redefinition of unused name from line n (F811)
Michael Fritch [Wed, 3 Feb 2021 14:06:16 +0000 (07:06 -0700)]
mgr/cephadm: Redefinition of unused name from line n (F811)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 80990f5bdc8eebeffa0cbb8e2c525ad268ed5cea)

4 years agomgr/cephadm: Module imported but unused (F401)
Michael Fritch [Wed, 3 Feb 2021 14:06:06 +0000 (07:06 -0700)]
mgr/cephadm: Module imported but unused (F401)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2e2b419dafd4738553e85a905151ba1e6ffce13d)

4 years agomgr/cephadm: Local variable name is assigned to but never used (F841)
Michael Fritch [Tue, 2 Feb 2021 17:42:44 +0000 (10:42 -0700)]
mgr/cephadm: Local variable name is assigned to but never used (F841)

add error handling when `CompletedProcess` fails during rgw relam,
zongroup create et al.

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6658c53d6f651373f0d9c2559b3524815604d30a)

4 years agomgr/cephadm: Local variable name is assigned to but never used (F841)
Michael Fritch [Tue, 2 Feb 2021 17:42:36 +0000 (10:42 -0700)]
mgr/cephadm: Local variable name is assigned to but never used (F841)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 97c6a8564ecbc5f39a1907d132b6646f55588326)

4 years agomgr/cephadm: Line break occurred after a binary operator (W504)
Michael Fritch [Thu, 28 Jan 2021 20:16:04 +0000 (13:16 -0700)]
mgr/cephadm: Line break occurred after a binary operator (W504)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4e4d4c5e4b6118c1629083b0bfca0fb00b500271)

4 years agomgr/cephadm: Trailing whitespace (W291)
Michael Fritch [Thu, 28 Jan 2021 02:42:33 +0000 (19:42 -0700)]
mgr/cephadm: Trailing whitespace (W291)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 8e093a5993dac6ce77efb7fd9ada1671e249b475)

4 years agomgr/cephadm: undefined name 'channel' (F821)
Michael Fritch [Thu, 28 Jan 2021 19:50:42 +0000 (12:50 -0700)]
mgr/cephadm: undefined name 'channel' (F821)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 867c7423e505e3ce06bbac3a18e64fa2b4db3f14)

4 years agomgr/cephadm: use ==/!= to compare constant literals (F632)
Michael Fritch [Thu, 28 Jan 2021 18:24:57 +0000 (11:24 -0700)]
mgr/cephadm: use ==/!= to compare constant literals (F632)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b40d28ce900fc4eb88be2480ae584c095120a51a)

4 years agomgr/cephadm: f-string is missing placeholders (F541)
Michael Fritch [Thu, 28 Jan 2021 19:43:10 +0000 (12:43 -0700)]
mgr/cephadm: f-string is missing placeholders (F541)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 22d6a1b7d8ce5530ef67f2ea8b1a9f9681dedf3a)

4 years agomgr/cephadm: Do not use variables named 'I', 'O', or 'l' (E741)
Michael Fritch [Thu, 28 Jan 2021 20:05:56 +0000 (13:05 -0700)]
mgr/cephadm: Do not use variables named 'I', 'O', or 'l' (E741)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fb73d8e5c21c6ff448d1cc28b7c4900456022b82)

4 years agomgr/cephadm: Do not assign a lambda expression, use a def (E731)
Michael Fritch [Thu, 28 Jan 2021 19:17:22 +0000 (12:17 -0700)]
mgr/cephadm: Do not assign a lambda expression, use a def (E731)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit e26db8c21063f53d2fe471d39e6cf4037e3ead15)

4 years agomgr/cephadm: Do not use bare except, specify exception instead (E722)
Michael Fritch [Thu, 28 Jan 2021 19:13:39 +0000 (12:13 -0700)]
mgr/cephadm: Do not use bare except, specify exception instead (E722)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 9b0a3e623984605bbcbcd82d38a7ccb6106b1d2b)

4 years agomgr/cephadm: Comparison to true should be 'if cond is true:' or 'if cond:' (E712)
Michael Fritch [Thu, 28 Jan 2021 18:57:16 +0000 (11:57 -0700)]
mgr/cephadm: Comparison to true should be 'if cond is true:' or 'if cond:' (E712)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 5136f4b1d0e6426fc8d3b2b2004ba32b58f211d3)

4 years agomgr/cephadm: Multiple statements on one line (def) (E704)
Michael Fritch [Thu, 28 Jan 2021 03:18:40 +0000 (20:18 -0700)]
mgr/cephadm: Multiple statements on one line (def) (E704)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 1738e22ec0b08ba225de55bf3772b6242c6cef2f)

4 years agomgr/cephadm: statement ends with a semicolon (E703)
Michael Fritch [Fri, 5 Feb 2021 18:13:40 +0000 (11:13 -0700)]
mgr/cephadm: statement ends with a semicolon (E703)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bed9be73ee8fdbb487b18bf5fc0f149174ca0497)

4 years agomgr/cephadm: Expected 1 blank line before a nested definition (E306)
Michael Fritch [Thu, 28 Jan 2021 03:16:05 +0000 (20:16 -0700)]
mgr/cephadm: Expected 1 blank line before a nested definition (E306)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d76324ea38cdcf9f1dac387f60bf0bec008b3ded)

4 years agomgr/cephadm: Expected 2 blank lines, found 0 (E302)
Michael Fritch [Thu, 28 Jan 2021 02:52:03 +0000 (19:52 -0700)]
mgr/cephadm: Expected 2 blank lines, found 0 (E302)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7e73af4b9880f971129b213a55d083bea2d9c342)

4 years agomgr/cephadm: at least two spaces before inline comment (E261)
Michael Fritch [Fri, 5 Feb 2021 18:13:05 +0000 (11:13 -0700)]
mgr/cephadm: at least two spaces before inline comment (E261)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 46a322ba018760610706eae92e7bd28e9b325029)

4 years agomgr/cephadm: multiple spaces after ':' (E241)
Michael Fritch [Thu, 28 Jan 2021 19:45:10 +0000 (12:45 -0700)]
mgr/cephadm: multiple spaces after ':' (E241)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ae76ba8a548b5543c645edc1195c45caf639eeb2)

4 years agomgr/cephadm: Missing whitespace after ',', ';', or ':' (E231)
Michael Fritch [Thu, 28 Jan 2021 20:07:58 +0000 (13:07 -0700)]
mgr/cephadm: Missing whitespace after ',', ';', or ':' (E231)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit cf4d666573299fe3d4614ae96b0e96a8acf7bdab)

4 years agomgr/cephadm: Missing whitespace around arithmetic operator (E226)
Michael Fritch [Thu, 28 Jan 2021 19:23:48 +0000 (12:23 -0700)]
mgr/cephadm: Missing whitespace around arithmetic operator (E226)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ef45cc50b583797b13c29f656dd65f21763297b6)

4 years agomgr/cephadm: Missing whitespace around operator (E225)
Michael Fritch [Thu, 28 Jan 2021 20:09:23 +0000 (13:09 -0700)]
mgr/cephadm: Missing whitespace around operator (E225)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 17c7c4ae7306198cc57215ec474fbaa9d425fbe1)

4 years agomgr/cephadm: Continuation line under-indented for visual indent (E128)
Michael Fritch [Thu, 28 Jan 2021 18:55:43 +0000 (11:55 -0700)]
mgr/cephadm: Continuation line under-indented for visual indent (E128)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 918eec7d1be1eedc5305f5f9d928351fbfc19437)

4 years agomgr/cephadm: Continuation line over-indented for hanging indent (E126)
Michael Fritch [Thu, 28 Jan 2021 03:15:03 +0000 (20:15 -0700)]
mgr/cephadm: Continuation line over-indented for hanging indent (E126)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b04fb5f2626d2621c666c7ae1bb00befa3d96dd3)

4 years agomgr/cephadm: closing bracket does not match visual indentation (E124)
Michael Fritch [Thu, 4 Feb 2021 19:52:31 +0000 (12:52 -0700)]
mgr/cephadm: closing bracket does not match visual indentation (E124)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bb28ad2b5b2dd4b93eb88c21131b1aa3a9e404f1)

4 years agomgr/cephadm: Continuation line missing indentation or outdented (E122)
Michael Fritch [Thu, 28 Jan 2021 03:08:32 +0000 (20:08 -0700)]
mgr/cephadm: Continuation line missing indentation or outdented (E122)

also resolves:
continuation line unaligned for hanging indent (E131)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b025081f8e016414b0698e5cd79740ea5bdc13c3)

4 years agomgr/cephadm: Continuation line under-indented for hanging indent (E121)
Michael Fritch [Thu, 28 Jan 2021 03:02:27 +0000 (20:02 -0700)]
mgr/cephadm: Continuation line under-indented for hanging indent (E121)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 144ff982d263ccbf25aa12412fca6bc4382a9a3f)

4 years agomgr/cephadm: Indentation is not a multiple of four (comment) (E114)
Michael Fritch [Thu, 28 Jan 2021 03:00:16 +0000 (20:00 -0700)]
mgr/cephadm: Indentation is not a multiple of four (comment) (E114)

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4005d606c0bc7ea4e8f06983db8753a3bf341a32)

4 years agomgr/cephadm: introduce flake8
Michael Fritch [Tue, 26 Jan 2021 19:02:38 +0000 (12:02 -0700)]
mgr/cephadm: introduce flake8

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit baacb36be008d61cc21bb0f94a005e2d6a510866)

Conflicts:
src/pybind/mgr/tox.ini

4 years agocephadm: do not cast subnet to unicode
Kefu Chai [Mon, 8 Feb 2021 08:27:44 +0000 (16:27 +0800)]
cephadm: do not cast subnet to unicode

this change addresses a regression introduced by
fe4f4402fbcd87667613640f2808d5d0e07e749d, which was tested before
abd9287db0e4f4f7873864119f5ce62519af1d48 got merged. and the former was
merged after abd9287db0e4f4f7873864119f5ce62519af1d48. hence the
regression.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b5851648771dcd93251114021005f44a66e602d5)

4 years agomgr/cephadm: cleanup daemon keyring on failed daemon deploy
Sebastian Wagner [Wed, 3 Feb 2021 15:24:25 +0000 (16:24 +0100)]
mgr/cephadm: cleanup daemon keyring on failed daemon deploy

Fixes: https://tracker.ceph.com/issues/48164
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f200cb26cc0d796c3af583e1d2bf72ec65213cd6)

4 years agomgr/cephadm:update test_orchestrator func signature
Paul Cuzner [Thu, 4 Feb 2021 23:05:05 +0000 (12:05 +1300)]
mgr/cephadm:update test_orchestrator func signature

The available signature has changed to 3 parms, so this
patch is needed to ensure it conforms.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit f6ccd2ebf24d4c2cf6ff073e0ccc735115b758de)

4 years agomgr/cephadm:toleration fix for API consistency in dashboard
Paul Cuzner [Wed, 3 Feb 2021 19:57:16 +0000 (08:57 +1300)]
mgr/cephadm:toleration fix for API consistency in dashboard

The function signatire change to "available" needed a
patch to the dashboard>orchestratior interface. This patch
just tolerates the change - if or how to consume the
additional data from "available" is yet to be deternined.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit e36e1d20acf4534133bcc14dab62cc5465667a62)

4 years agomgr/cephadm:updates to support orch status changes
Paul Cuzner [Wed, 3 Feb 2021 01:45:15 +0000 (14:45 +1300)]
mgr/cephadm:updates to support orch status changes

The available function definition has been updated in
cephadm/rook and Orchestrator base class to provide a
module specific dictionary holding any specific info that
would be pertinent to share with the user. This in turn
changed the _status method for cephadm, but the UX
remains the same (i.e. admin must use --detail to see
the worker pool size).

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 81caa2643ec4a36d6d6185a4fb215eaf4b532aef)

Conflicts:
src/pybind/mgr/rook/module.py

4 years agoPendingReleaseNotes: mgr/pg_autoscaler 39637/head
Kamoltat [Thu, 11 Feb 2021 15:44:09 +0000 (15:44 +0000)]
PendingReleaseNotes: mgr/pg_autoscaler

Added details describing the changes that
occured after merging #39248 into target
upstream.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit f9f18bb5df63cad9715fb7c4e2299420ff52269d)