]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Sage Weil [Tue, 2 Feb 2021 23:07:36 +0000 (17:07 -0600)]
mgr/cephadm: use datetime_now() for last_facts_update
Be consistent!
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
821f7e9d5b5a2d4725c5cea2987e601e07e83558 )
Sage Weil [Tue, 2 Feb 2021 19:46:44 +0000 (13:46 -0600)]
mgr/cephadm: track last_device_change
Keep track of when the device inventory and/or state *changes*.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2b7d8e16309b12273d2e65ce638c9588528ee1f0 )
Sage Weil [Tue, 2 Feb 2021 18:01:32 +0000 (12:01 -0600)]
mgr/cephadm: track last_applied by host for osd specs
For each host, note when we last applied each osdspec. Log the start
time, not the end time.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
197a8ba22ff30ccf9498bbc14b7a3897e48e1220 )
Sebastian Wagner [Wed, 3 Feb 2021 12:56:09 +0000 (13:56 +0100)]
mgr/cephadm: on ssh connection error, advice chmod 0600
Fixes: https://tracker.ceph.com/issues/46551
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
e3fa11f40ebc28782c65b4f8ee5d69237eb63d7d )
Sage Weil [Fri, 19 Feb 2021 21:43:05 +0000 (16:43 -0500)]
Merge PR #39563 into pacific
* refs/pull/39563/head:
qa/valgrind.supp: suppress free[] warnings
qa/suites: do not use notcmalloc flavor
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Fri, 19 Feb 2021 17:04:58 +0000 (09:04 -0800)]
Merge pull request #39433 from liewegas/pr-39404-pacific
pacific: qa/suites/upgrade/octopus-x/stress-split
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Sage Weil [Thu, 18 Feb 2021 18:04:16 +0000 (12:04 -0600)]
qa/valgrind.supp: suppress free[] warnings
This is apparently not going to get fixed any time soon.
https://github.com/gperftools/gperftools/issues/792
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
28fe149ea276693b47cc4d600780971f723672e4 )
Sage Weil [Thu, 18 Feb 2021 22:00:59 +0000 (16:00 -0600)]
qa/suites: do not use notcmalloc flavor
teuthology now knows how to run valgrind against a tcmalloc binary
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
dc64ccf063f1ddf843fae4d2621ec25187f56135 )
# Conflicts:
# qa/suites/rgw/verify/validater/valgrind.yaml
- conflict on commented out line
Ernesto Puerta [Thu, 18 Feb 2021 16:07:47 +0000 (17:07 +0100)]
Merge pull request #39442 from rhcs-dashboard/wip-49270-pacific
pacific: mgr/dashboard: delete EOF when reading passwords from file
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 20:15:31 +0000 (12:15 -0800)]
Merge pull request #39485 from liewegas/pr-39282-pacific
pacific: osdc,librados: set FULLY_TRY flag by ioctx, not globally
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Wed, 17 Feb 2021 19:50:50 +0000 (11:50 -0800)]
Merge pull request #39484 from liewegas/pr-39283-pacific
pacific: qa/suites/rados/dashboard: whitelist TELEMETRY_CHANGED
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Tue, 16 Feb 2021 22:56:10 +0000 (14:56 -0800)]
Merge pull request #39500 from liewegas/pr-39496-pacific
pacific: mgr/cephadm: fix host refresh
Reviewed-by: David Galloway <dgallowa@redhat.com>
Ernesto Puerta [Tue, 16 Feb 2021 19:49:07 +0000 (20:49 +0100)]
Merge pull request #39425 from rhcs-dashboard/wip-49254-pacific
pacific: mgr/dashboard: Automatically refresh crush map metadata table
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yuri Weinstein [Tue, 16 Feb 2021 19:13:49 +0000 (11:13 -0800)]
Merge pull request #39310 from liewegas/pr-39213-pacific
pacific: qa/suites/upgrade/octopus-x/parallel: include cephfs in upgrade cluster
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Sage Weil [Tue, 16 Feb 2021 17:07:08 +0000 (11:07 -0600)]
mgr/cephadm: fix host refresh
Fixes: 01f60cf4e0a751c314120c02956d4ff941eb71b4
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9df5a6d73ed21b394c01afe6c9800b6e50737c90 )
Patrick Donnelly [Mon, 15 Feb 2021 23:15:35 +0000 (15:15 -0800)]
Merge PR #39437 into pacific
* refs/pull/39437/head:
qa: Fix a few mgr/volume test cases
Reviewed-by: Ramana Raja <rraja@redhat.com>
Sage Weil [Wed, 3 Feb 2021 21:46:22 +0000 (16:46 -0500)]
osdc/Objecter: remove pool_full_try interface
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
e468fc0d3b8c5f4d7713e4015d4d7ccbfa5bccf9 )
Sage Weil [Wed, 3 Feb 2021 21:44:11 +0000 (16:44 -0500)]
librados: set FULL_TRY flag on ioctx, not objecter
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a8bed67ad17f8d9114888cd1eba81c05555cfa73 )
Sage Weil [Wed, 3 Feb 2021 21:43:53 +0000 (16:43 -0500)]
librados/IoCtxImpl: pass extra_op_flags everywhere
Unused, so far...
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
68564836ff637ce017755abfa10c1856296dc51d )
Sage Weil [Wed, 3 Feb 2021 23:36:29 +0000 (17:36 -0600)]
qa/suites/rados/dashboard: whitelist TELEMETRY_CHANGED
test_enable_module_empty_license fiddles with this.
Fixes: https://tracker.ceph.com/issues/48990
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
bf5646704c3e565efbb480070e7df5745c89c588 )
Josh Durgin [Mon, 15 Feb 2021 17:52:46 +0000 (09:52 -0800)]
Merge pull request #39232 from liewegas/pr-39212-pacific
pacific: cpatch: updates
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Jan Fajerski [Mon, 15 Feb 2021 09:14:49 +0000 (10:14 +0100)]
Merge PR #39222 into pacific
* refs/pull/39222/head:
ceph-volume: Fix usage of is_lv
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Yuri Weinstein [Fri, 12 Feb 2021 18:14:48 +0000 (10:14 -0800)]
Merge pull request #39179 from neha-ojha/wip-49058-pacific
pacific: qa/*/thrash_cache_writeback_proxy_none.yaml: disable writeback overlay tests
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Fri, 12 Feb 2021 16:17:07 +0000 (08:17 -0800)]
Merge pull request #39228 from ifed01/wip-ifed-fix-pin-pacific
pacific: os/bluestore: fix a bug causing unexpected Onode's unpinned state.
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Fri, 12 Feb 2021 16:16:25 +0000 (08:16 -0800)]
Merge pull request #39218 from k0ste/wip-49006-pacific
pacific: mgr: update mon metadata when monmap is updated
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Fri, 12 Feb 2021 16:15:31 +0000 (08:15 -0800)]
Merge pull request #39214 from neha-ojha/wip-49087-pacific
pacific: qa/suites/upgrade/nautilus-x-singleton: pin to 18.04
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Fri, 12 Feb 2021 16:15:03 +0000 (08:15 -0800)]
Merge pull request #39180 from neha-ojha/wip-48986-pacific
pacific: An empty bucket or OSD is not an error
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Sebastian Wagner [Fri, 12 Feb 2021 15:20:55 +0000 (16:20 +0100)]
Merge pull request #39378 from sebastian-philipp/pacific-backport-39172-39097-39095-39143-39106-39200-38911-39069-39208-38927-
pacific: cephadm backport February (1)
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Alfonso Martínez [Tue, 9 Feb 2021 10:17:52 +0000 (11:17 +0100)]
mgr/dashboard: delete EOF when reading passwords from file
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit
caeadf1397db00c6b7ba218b1910508099802e39 )
Jason Dillaman [Fri, 12 Feb 2021 13:15:47 +0000 (08:15 -0500)]
Merge pull request #39428 from dillaman/wip-49253-pacific
pacific: librbd: correct incremental deep-copy object-map inconsistencies
Reviewed-by: Mykola Golub <mgolub@suse.com>
Jason Dillaman [Fri, 12 Feb 2021 13:15:25 +0000 (08:15 -0500)]
Merge pull request #39316 from petrutlucian94/wip-49155-pacific
pacific: Windows RBD fixes
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 12 Feb 2021 13:15:00 +0000 (08:15 -0500)]
Merge pull request #39305 from dillaman/wip-librbd-backports-pacific-3
pacific: librbd: miscellaneous backports
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Kotresh HR [Fri, 5 Feb 2021 18:05:22 +0000 (23:35 +0530)]
qa: Fix a few mgr/volume test cases
Recovering dirty auth metadata file might not retain the order,
fixed the comparison in 'test_recover_auth_metadata_during_authorize'
and 'test_recover_auth_metadata_during_deauthorize'.
Fixes: https://tracker.ceph.com/issues/49192
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit
4952d2132ffd92594e749a652970d5d9415c721d )
Sage Weil [Thu, 4 Feb 2021 18:56:49 +0000 (12:56 -0600)]
qa/suites/upgrade/octopus-x/stress-split: cephadm-based stress-split upgrade
Apply stress during upgrade, and also pause partway and do some
thrashing.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
ec90e9b82c540bbdc7194e2cf3c5d864ab918695 )
Ilya Dryomov [Tue, 9 Feb 2021 13:04:39 +0000 (14:04 +0100)]
librbd: journal::StandardPolicy template defaults to ImageCtx
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
b90ee634fa2aca4d1223ee19f43d026917aa969d )
Ilya Dryomov [Mon, 8 Feb 2021 16:01:47 +0000 (17:01 +0100)]
librbd: don't hold owner_lock for validate_image_removal()
handle_exclusive_lock() and handle_shut_down_exclusive_lock() call
validate_image_removal() without owner_lock held, so holding it in
shut_down_exclusive_lock() appears to be redundant.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
b1bf3eaef200fb5d1aa847cad14936f2bbbf7b64 )
Ilya Dryomov [Sun, 7 Feb 2021 14:09:24 +0000 (15:09 +0100)]
librbd: treat EROFS as expected in handle_acquire_lock()
If the peer refuses to release exclusive lock (e.g. in case automatic
exclusive lock transitions are disabled), EROFS is retured. Suppress
a rather confusing "Read-only file system" error message -- this case
is no different from EBUSY or EAGAIN.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
2f82c98f5a542a69057f3e78d232380eee14eff7 )
Ilya Dryomov [Sun, 7 Feb 2021 12:46:15 +0000 (13:46 +0100)]
librbd: refuse to release exclusive lock when removing
Commit
25c2ffe145be ("librbd: acquire exclusive lock from peer when
removing") changed PreRemoveRequest to request exclusive lock from the
peer instead of giving up and proceeding without exclusive lock. This
caused one of the test cases that sometimes runs concurrent "rbd rm"
against the same image to fail intermittently, most often on assert
ceph_assert(image_ctx.exclusive_lock == nullptr ||
image_ctx.exclusive_lock->is_lock_owner());
because exclusive lock is now automatically transitioned to another
"rbd rm" on its request.
The root cause is older and probably goes back to when synchronous
librbd::remove() which held owner_lock across all operations including
trim_image() was converted to a set of state machines. Since then, any
peer that requests exclusive lock (instead of trying once and backing
off) is able to mess with image removal.
Install StandardPolicy to disable automatic exclusive lock transitions
during image removal.
Fixes: https://tracker.ceph.com/issues/49226
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
707907ea3f6ff39968666af4ba718ceef8cd8953 )
Ilya Dryomov [Mon, 8 Feb 2021 15:49:35 +0000 (16:49 +0100)]
librbd: templatize exclusive_lock::StandardPolicy
This will be used by expect_set_exclusive_lock_policy() in
test_mock_PreRemoveRequest.cc.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
948f3c761dce4678b38d22e240da06434dbe10e4 )
Mahati Chamarthy [Wed, 3 Feb 2021 15:03:50 +0000 (20:33 +0530)]
librbd/cache/pwl: Fix user request completion in
... persist_on_flush mode
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
(cherry picked from commit
d8eb9391c9dd7425053bd91f79eea103ae90cc93 )
Jason Dillaman [Mon, 8 Feb 2021 16:53:28 +0000 (11:53 -0500)]
rbd-mirror: don't prune older mirror snapshots when pruning incomplete snapshot
Since we normally prune in order, we need to ensure that we don't prune older
snapshots when we need to delete an incomplete mirror snapshot since the
older snapshot might be the only remaining mirror snapshot.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
7ba9214ea5b73d0436af6c2896abf4836d741de9 )
Jason Dillaman [Mon, 8 Feb 2021 15:13:35 +0000 (10:13 -0500)]
qa/workunits/rbd: show snapshot deltas during stress test failure
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
f079116e87161b55acaa08c55bf8b8e79cee8670 )
Jason Dillaman [Fri, 5 Feb 2021 18:33:16 +0000 (13:33 -0500)]
librbd/deep-copy: compute parent overlap before computing zero extents
An object might not overlap with its parent so avoid injecting an empty
object to hide the non-existent parent.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
f1c6147623fc7876f7f9cc7beda98d1f7edf9e68 )
Jason Dillaman [Tue, 2 Feb 2021 20:28:48 +0000 (15:28 -0500)]
qa/suites/rbd: add snapshot-based mirroring stress test
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
094bfeaf8efe1c4794a2b301314eddedfa5246f9 )
Jason Dillaman [Fri, 29 Jan 2021 15:44:38 +0000 (10:44 -0500)]
librbd/deep_copy: skip snap list if object is known to be clean
If the fast-diff indicates that the destination object should exist
and that it hasn't changed, there shouldn't be a need to issue the
snap list operation. Instead, just update the destination object map
to indicate the existence of the object.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
235b27a8f08a82fc46efa8b6a3e5cb8a5276956b )
Jason Dillaman [Fri, 29 Jan 2021 02:42:09 +0000 (21:42 -0500)]
librbd/deep_copy: object-copy state machine must update object map
If there was no data to copy, the object-copy state machine was bypassing
the object-map update states and prematurely completing. Since the
object-map is default-initialized to all non-existent objects, this results
in incorrect state for OBJECT_EXISTS_CLEAN objects.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
ca0b9bfc28ef7287ca139ca9640c876223eda87b )
Jason Dillaman [Wed, 3 Feb 2021 18:21:34 +0000 (13:21 -0500)]
librbd/io: track object non-existence when computing snapshot deltas
Re-use the existing DNE state to track whether or not the object
already exists when computing snapshot deltas from an arbitrary
set of snapshots. Previously, the non-existence of the object was
only computed for snap id 0 for tracking whiteouts. In a future
commit, the deep-copy object-copy state machine will be able to
properly update the object-map state to indicate exists clean
vs non-existent state.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
89d2a13db67945bfd7c6d6fe79584f00574b8ba3 )
Jason Dillaman [Wed, 3 Feb 2021 21:23:20 +0000 (16:23 -0500)]
librbd/io: minor reorganization for object list-snaps
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
a75c8a0abdb43a60dd7eeb3b405f6b30481cf321 )
Jason Dillaman [Wed, 3 Feb 2021 15:13:28 +0000 (10:13 -0500)]
librbd/io: only track initial diff extents if no diffs exists
The purpose of the initial diff extents ({0, 0}) was to help track
whether or not objects exists for read-from-parent / whiteout
tracking. Once we have at least one set of diffs on the object, we
actually have enough information to know about the object state.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
688cbf3da38025b48be1e308f79170c40f0760d3 )
Jason Dillaman [Thu, 28 Jan 2021 23:30:16 +0000 (18:30 -0500)]
librbd/object_map: diff state machine should track object existence
The deep-copy snapshot-create state machine initializes the object-map
state to non-existent for all objects. There was an assumption that the
deep-copy object-copy state machine would always update the object map
but that was being skipped for clean objects as an optimization. This
change will support a future commit to run the object-copy state machine
for existing objects.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
b81cd2460de748c71210520f8c819895f257f0c7 )
Nizamudeen A [Tue, 19 Jan 2021 12:35:43 +0000 (18:05 +0530)]
mgr/dashboard: Automatically refresh the crush map metadata table
If we make any change to the osd crush map like do an osd crush reweight from cli, for that change to be reflected on metadata table we need to reload the entire page. Instead this PR takes care of auto refreshing the tree view.
Fixes: https://tracker.ceph.com/issues/48922
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit
bc8562ef2a17b78e80bd4e1272d3fd1a512249bb )
Guillaume Abrioux [Thu, 11 Feb 2021 07:49:03 +0000 (08:49 +0100)]
Merge pull request #39272 from jan--f/wip-49142-pacific
pacific: ceph-volume: add some flexibility to bytes_to_extents
Neha Ojha [Thu, 11 Feb 2021 02:06:18 +0000 (18:06 -0800)]
Merge pull request #39403 from neha-ojha/wip-49228-pacific
pacific: Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"
Reviewed-by: Sage Weil <sage@redhat.com>
Neha Ojha [Tue, 9 Feb 2021 22:10:19 +0000 (22:10 +0000)]
Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"
This reverts commit
6d7ba1180e66df2d6dee3a01fe3e12cafaa6fe8c .
Fixes: https://tracker.ceph.com/issues/49228
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
3ced450cdd7a1f4083a19dd29b2d27e03883ab52 )
Sebastian Wagner [Fri, 29 Jan 2021 10:10:38 +0000 (11:10 +0100)]
mgr/cephadm: Add strings to assert statements
This helps with: https://tracker.ceph.com/issues/48981
Looks like there is an assert somewhere:
```
Error EINVAL: Traceback (most recent call last):
File "/usr/share/ceph/mgr/mgr_module.py", line 1269, in _handle_command
return self.handle_command(inbuf, cmd)
...snip...
File "/usr/share/ceph/mgr/orchestrator/module.py", line 550, in _list_services
raise_if_exception(completion)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 653, in raise_if_exception
raise e
AssertionError
```
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
06f4d84166f9c0761ace002fa2091dcd1d78f2d6 )
Sebastian Wagner [Fri, 15 Jan 2021 12:13:35 +0000 (13:13 +0100)]
mgr/cephadm: try again calling ceph-volume without --filter-for-batch
Fixes: https://tracker.ceph.com/issues/48870
This deals with a cephadm upgrade issue:
1. user calls `ceph orch upgrade`
2. mgr/cephadm calls `ceph orch config set mgr.x container_image <new-container>`
3. standby mgr gets upgraded
4. mgr failover to new mgr
5. mgr/cephadm calls `_refresh_host_devices`
6. `_refresh_host_devices` calls` ceph orch config get osd container_image`.
But this returns the old image
7. `_refresh_host_devices` calls `ceph-volume ... --filter-for-batch`
with an image that doesn't support `filter-for-batch`
The idea is to simply retiry calling ceph-volume inventory without `--filter-for-batch`
(also removed `out` being used without being declared)
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
ede3d6d147dd7b99d37aee5c5fb9340f2878db18 )
Sebastian Wagner [Mon, 1 Feb 2021 16:08:28 +0000 (17:08 +0100)]
python-common: Set disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
582f7c1657766579acc69512bf7a90088e5618a2 )
Sebastian Wagner [Mon, 1 Feb 2021 16:06:57 +0000 (17:06 +0100)]
python-common: removed unused exeption.py
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
16dfa60b9feff3eb3a2748c4e378d5abd8e3378c )
Sebastian Wagner [Mon, 1 Feb 2021 15:13:19 +0000 (16:13 +0100)]
python-common: Type annotation for ServiceSpec.from_json
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit
7a297262aa1ec080f2264f5b16747cdd740d446d )
Sage Weil [Sat, 30 Jan 2021 22:35:21 +0000 (16:35 -0600)]
mgr/cephadm/upgrade: tolerate pre-pacific upgrade state
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
b0ff67b7ff321ed6e73ab19ea6282ebc75546b19 )
Sage Weil [Tue, 26 Jan 2021 22:10:13 +0000 (16:10 -0600)]
mgr/cephadm/upgrade: scale down MDS cluster(s) for major version upgrades
For octopus -> pacific, as with other recent releases, we need to scale
down the MDS cluster(s) to a single daemon before upgrading. (This is
because the MDS intra-cluster protocols aren't fully versioned.)
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
604d2636d7744b742c162df722ed53ec68a9084e )
Sage Weil [Thu, 28 Jan 2021 15:02:27 +0000 (09:02 -0600)]
mgr/cephadm: fix capitalization, level; drop elipses of log msgs
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
1e60a103709aec5ddac8b12a44b9d0d646d2a682 )
Sage Weil [Wed, 27 Jan 2021 14:54:00 +0000 (08:54 -0600)]
mgr/cephadm/upgrade: match against any repo_digest, not image_id
The image id can vary across hosts and (most notably) docker vs podman.
Instead, use the repo_digest as an image identifier.
Unfortunately, a single image may have multiple digests, even within the
same registry, so keep a list of the digests for the image we are
upgrading to, and ensure that each container has a digest that matches at
least one of them.
This allows upgrade to proceed in mixed docker+podman clusters. However,
it does not yet address a cluster with mixed CPU architectures, because
the container image will have different digest(s) for each architecture
build.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
fe72946729dada7646863b17c06d90abd82d61ad )
Sage Weil [Wed, 27 Jan 2021 14:12:00 +0000 (08:12 -0600)]
cephadm: return repo_digests (plural) in pull/inspect output
When we inspect a (pulled) image, return all of the repo digests. Update
the mgr/cephadm code accordingly to match.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
e9333972c1b84ebdfe43c8f686d74d4c859eedf1 )
Sage Weil [Wed, 27 Jan 2021 14:02:25 +0000 (08:02 -0600)]
mgr/cephadm: include container_image_digests in inventory
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
9f61c94ae7d56f0b12d753642e031fcd5cd9d145 )
Sage Weil [Wed, 27 Jan 2021 13:50:14 +0000 (07:50 -0600)]
cephadm: include image_digests list in 'ls' output
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
0826c45e0cb5d60fcf8cd71cd14edd34a6997cd4 )
Sage Weil [Tue, 26 Jan 2021 19:11:13 +0000 (13:11 -0600)]
vstart.sh: only extract first container digest
Otherwise the container_image value includes the same hash twice, separated
by a newline, causing all sorts of confusion.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
6741082eb3e03db608899a8a5fef44fb05d356b7 )
Sage Weil [Tue, 26 Jan 2021 15:24:54 +0000 (09:24 -0600)]
mgr/cephadm: move release -> major translation to helper
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
43ac324c3817b378beb9fe12437565ddbecb5370 )
Sage Weil [Tue, 26 Jan 2021 15:24:38 +0000 (09:24 -0600)]
mgr/cephadm/upgrade: tolerate old upgrade_state.target_versoin
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a3c9df91975708f42cd75266e840f0b0b019e7c7 )
Sage Weil [Mon, 25 Jan 2021 23:21:05 +0000 (17:21 -0600)]
mgr/cephadm/upgrade: set require-osd-release when done with OSDs
After we've upgraded all of the osds, complete the major version upgrade
for core RADOS by setting the OSD cluster min.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
8abd45c4c95a58ae67c29905a3c4f6c77a7383e9 )
Sage Weil [Mon, 25 Jan 2021 23:19:59 +0000 (17:19 -0600)]
mgr: add lookup_release_name(int) to mgr interface
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
ceac0861434ae761d8866889980dcf95d1f21a93 )
Sage Weil [Mon, 25 Jan 2021 22:47:16 +0000 (16:47 -0600)]
mgr/cephadm: verify container image version after we pull it
Apply the version checks to make sure this upgrade/downgrade is possible.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
5147d0f9305d97bf251671da80200f92536d4862 )
Sage Weil [Mon, 25 Jan 2021 22:46:43 +0000 (16:46 -0600)]
mgr/cephadm: only save version portion of version string
We don't want to 'ceph version ' prefix or sha1 suffix. This matches us
up with 'orch upgrade start'
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
dcaf514bbf1ac19c0ea9f5bfa724bca76ff2d126 )
Sage Weil [Mon, 25 Jan 2021 22:32:45 +0000 (16:32 -0600)]
cephadm: fix 'inspect' and 'pull'
With podman 2.2.1 I would get
ERRO[0000] Error printing inspect output: template: all inspect:1: function "json" not defined
Removing the 'json' portion of the query resolves it.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
d31bed79411ca493ec48eeed4e9cbb7ad92295c3 )
Sage Weil [Mon, 25 Jan 2021 20:00:04 +0000 (14:00 -0600)]
mgr/cephadm/upgrade: implement N-2 version checks on upgrade start
- Prevent major upgrades that span > 2 releases
- Prevent downgrades to an rc or dev release
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
95dd015bce4153e3c47adac89936a300a0dd3152 )
Paul Cuzner [Tue, 2 Feb 2021 02:42:00 +0000 (15:42 +1300)]
cephadm:updates following bootstrap code clean up
Patch to align to new(ish) bootstrap code changes
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
804f34e475be699807f565d48ecd13320d246782 )
Paul Cuzner [Thu, 28 Jan 2021 03:35:57 +0000 (16:35 +1300)]
cephadm: updated doc text
The phrase "recommended" has been removed.
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
af00a333224df4e748e662efc2bd48593af25d26 )
Paul Cuzner [Sun, 17 Jan 2021 23:24:16 +0000 (12:24 +1300)]
cephadm: unit test updates for cluster-network handling
Tests updated to handle comma separated subnet lists
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
bbe4fd83c556efb35c16f6da0bf2b00e11a2b3c9 )
Paul Cuzner [Sun, 17 Jan 2021 23:20:23 +0000 (12:20 +1300)]
cephadm: updated handling of cluster_network parameter
Parameter now supports a comma separated list of subnets and
detects the ipv4 or ipv6 setting on the cluster network to enable
osds to bind correctly.
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
abd9287db0e4f4f7873864119f5ce62519af1d48 )
Paul Cuzner [Thu, 14 Jan 2021 22:08:48 +0000 (11:08 +1300)]
cephadm: install doc updated to include cluster-network parameter
Install guide updated to include a description of the --cluster-network
parameter. The text also links to the complete definition for cluster-network
on the rados/configuration/network-config-ref page.
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
509f171a08a82a3bc132739b43500064266b8e9e )
Paul Cuzner [Thu, 14 Jan 2021 20:56:15 +0000 (09:56 +1300)]
cephadm: unit test updates for new subnet function
Adds tests to validate the check_subnet function will work as
expected with various inputs
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
3f352c8af4ddedd6ae7fc69045e42d07dad261c2 )
Paul Cuzner [Thu, 14 Jan 2021 20:53:34 +0000 (09:53 +1300)]
cephadm: Add cluster network parameter to bootstrap cmd
Adds a --cluster-network parameter to the bootstrap cmd to
set the internal cluster network.
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit
c391ccb97f1bc81bf9c45e9b313078535b6ad4c5 )
Mitsumasa KONDO [Mon, 1 Feb 2021 09:44:49 +0000 (18:44 +0900)]
doc/mgr/orchestrator: Unify the content of command and yaml
Signed-off-by: Mitsumasa KONDO <kondo.mitsumasa@gmail.com>
(cherry picked from commit
53639c2b01c80806e8271a623ad21b40d89a6d73 )
Michael Fritch [Wed, 27 Jan 2021 15:35:01 +0000 (08:35 -0700)]
cephadm: use `apt-get` for package install/update
avoids errors during prepare-host:
```
apt: stderr WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
```
Fixes: https://tracker.ceph.com/issues/49032
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit
fa4706039cfece36815df46cd4452dc25448c340 )
Adam King [Thu, 28 Jan 2021 17:12:57 +0000 (12:12 -0500)]
cephadm: fix get_data_dir calls for ha-rgw daemons
calls need to use the context's data dir rather than daemon type
puts these calls in line with the other daemons
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
10214317aee39ecca5e7744b2ee7f528a4b80cb5 )
Kefu Chai [Wed, 27 Jan 2021 01:49:20 +0000 (09:49 +0800)]
cephadm: add fsid if --name is not specified
address an regression introduced by
9e81a5dfc5abdd8caeecf42faa17111fc309b8d3
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
556b34a6679fe617743a46f5a0cf78fcb46a39b2 )
Kefu Chai [Wed, 27 Jan 2021 01:46:18 +0000 (09:46 +0800)]
cephadm: add CephadmContext.__contains__()
more pythonic this way.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
b9fb0d9f3de9960ecdb8901cadcf5a735b3bc4d2 )
Sage Weil [Fri, 29 Jan 2021 15:45:09 +0000 (09:45 -0600)]
mgr/orchestrator: remove 'host' arg for 'orch ls'
The host arg is unused, and also missing from the documentation.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
38b3ca2a736995baf962997995f1334786c6e2cd )
Sebastian Wagner [Wed, 10 Feb 2021 11:33:38 +0000 (12:33 +0100)]
Merge pull request #39309 from liewegas/pr-39150-pacific
pacific: qa/suites/rados/cephadm/thrash: add cephadm thrashing test
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sebastian Wagner [Wed, 10 Feb 2021 11:32:54 +0000 (12:32 +0100)]
Merge pull request #39181 from liewegas/pr-39039-pacific
pacific: quincy feature, custom git_url for cephadm_branch
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Sebastian Wagner [Wed, 10 Feb 2021 11:31:35 +0000 (12:31 +0100)]
Merge pull request #39190 from liewegas/pr-39097-pacific
pacific: qa/workunits/cephadm/test_cephadm: fixes and cleanup
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Sebastian Wagner [Wed, 10 Feb 2021 11:31:07 +0000 (12:31 +0100)]
Merge pull request #39252 from sebastian-philipp/pacific-cephadm-daemon-base-devel-pacific
pacific: cephadm: use ceph/daemon-base:latest-pacific-devel image
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Ramana Raja [Mon, 8 Feb 2021 20:16:18 +0000 (15:16 -0500)]
Merge pull request #39038 from kotreshhr/fix-test-idempotency-test
pacific: pybind/ceph_volume_client: Fix test_idempotency test failure
Reviewed-by: Ramana Raja <rraja@redhat.com>
Ramana Raja [Mon, 8 Feb 2021 20:14:44 +0000 (15:14 -0500)]
Merge pull request #39109 from kotreshhr/wip-49027-pacific
pacific: mgr/volumes: Evict clients based on auth-IDs and subvolume path
Reviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein [Mon, 8 Feb 2021 18:31:57 +0000 (10:31 -0800)]
Merge pull request #39284 from neha-ojha/wip-49145-pacific
pacific: osd: fix the scrubber behavior on multiple preemption attempts
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Yuri Weinstein [Mon, 8 Feb 2021 18:31:06 +0000 (10:31 -0800)]
Merge pull request #39264 from neha-ojha/wip-49134-pacific
pacific: qa/*/test_envlibrados_for_rocksdb: use osd_client_message_cap to prevent slow requests
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Mon, 8 Feb 2021 18:30:39 +0000 (10:30 -0800)]
Merge pull request #39257 from ifed01/wip-ifed-fsck-spanning-blobs-pac
pacific: os/bluestore: detect and fix "zombie" spanning blobs using fsck.
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Yuri Weinstein [Mon, 8 Feb 2021 18:29:46 +0000 (10:29 -0800)]
Merge pull request #39247 from kamoltat/wip-ksirivad-pacific-backports
pacific: mgr/pg_autoscaler: avoid scale-down until there is pressure
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Mon, 8 Feb 2021 18:28:51 +0000 (10:28 -0800)]
Merge pull request #39237 from liewegas/pr-39206-pacific
pacific: mgr/MetricTypes: condition encoding on feature bits
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Ernesto Puerta [Mon, 8 Feb 2021 18:01:25 +0000 (19:01 +0100)]
Merge pull request #39102 from k0ste/wip-49021-pacific
pacific: mgr/dashboard: trigger alert if some nodes have a MTU different than the median value
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>