Kefu Chai [Fri, 6 Mar 2020 04:17:40 +0000 (12:17 +0800)]
qa/tasks/ceph.py: quote "<kind>" in command line
otherwise bash will intepret "kind" as a file when handling command like
```
sudo zgrep <kind> /var/log/ceph/valgrind/* /dev/null | sort | uniq
```
and try to feed its content to zgrep, and write the output of zgrep
to /var/log/ceph/valgrind/*. this is not the intended behavior. what we
what to do is to pass "<kind>" as an argument to zgrep, along with
the globbed files names which matches "/var/log/ceph/valgrind/*".
Sage Weil [Thu, 5 Mar 2020 16:42:26 +0000 (10:42 -0600)]
mgr/cephadm: make osd create on an existing LV idempotent
If we try to prepare an LV that was already prepared, ceph-volume will
return an error message and code. We want our osd create command to be
idempotent, though, so recognize the error string and continue.
This is an ugly hack, but quicker than changing ceph-volume behavior, and
it is sufficient to stop all of the teuthology failures.
The second part of this is that we have to deploy the daemon on OSDs that
are already prepared and already exist in our osdmap beforehand, but have
never started.
Works-around: https://tracker.ceph.com/issues/44313 Signed-off-by: Sage Weil <sage@redhat.com>
volumes/fs/async_cloner.py: note: In function "handle_clone_pending":
volumes/fs/async_cloner.py:71: error: "OpSmException" has no attribute "error"; maybe "errno"?
volumes/fs/async_cloner.py: note: In function "handle_clone_in_progress":
volumes/fs/async_cloner.py:139: error: "OpSmException" has no attribute "error"; maybe "errno"?
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:22:48 +0000 (08:22 -0700)]
mgr/volumes: remove unneeded assignment to `NoneType`
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "get_subvolume_object" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "SubvolumeBase")
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:59 +0000 (08:21 -0700)]
mgr/volumes: add missing OpSmException import
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "upgrade_legacy_subvolume" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:56: error: Name 'OpSmException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:54 +0000 (08:21 -0700)]
mgr/volumes: add missing error code
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "_load_supported_versions" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:35: error: Too few arguments for "VolumeException"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:36 +0000 (08:21 -0700)]
mgr/volumes: fixup format string args
fixes mypy errors:
volumes/fs/purge_queue.py:26: error: Cannot find replacement for positional format specifier 1
volumes/fs/async_cloner.py:37: error: Cannot find replacement for positional format specifier 1
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:30 +0000 (08:21 -0700)]
mgr/volumes: add missing `mgr` param
fixes mypy errors:
volumes/fs/operations/volume.py: note: In function "create_volume":
volumes/fs/operations/volume.py:216: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:223: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:224: error: Too few arguments for "remove_pool"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:25 +0000 (08:21 -0700)]
mgr/volumes: assert self.fs
fixes mypy errors:
volumes/fs/operations/volume.py: note: In member "disconnect" of class "Connection":
volumes/fs/operations/volume.py:94: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
volumes/fs/operations/volume.py:95: error: Item "None" of "Optional[Any]" has no attribute "shutdown"
volumes/fs/operations/volume.py: note: In member "abort" of class "Connection":
volumes/fs/operations/volume.py:105: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:18 +0000 (08:21 -0700)]
mgr/volumes: skip type checking on RTimer class
Unclear why mypy does not like this:
volumes/fs/operations/volume.py: note: In member "run" of class "RTimer":
volumes/fs/operations/volume.py:118: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "interval"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "function"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "args"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "kwargs"
volumes/fs/operations/volume.py:121: error: "RTimer" has no attribute "finished"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:20:01 +0000 (08:20 -0700)]
mgr/volumes: fix positional str formatting
fixes mypy error:
volumes/fs/operations/group.py: note: In function "create_group":
volumes/fs/operations/group.py:135: error: Not all arguments converted during string formatting
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:19:53 +0000 (08:19 -0700)]
mgr/volumes: place getters and setters next to each other
workaround for mypy issue:
https://github.com/python/mypy/issues/1465
fixes mypy errors:
volumes/fs/operations/group.py: note: In class "Group":
volumes/fs/operations/group.py:44: error: Name 'uid' already defined on line 36
volumes/fs/operations/group.py:44: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py:48: error: Name 'gid' already defined on line 40
volumes/fs/operations/group.py:48: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py: note: In function "open_group":
volumes/fs/operations/group.py:170: error: Property "uid" defined in "Group" is read-only
volumes/fs/operations/group.py:171: error: Property "gid" defined in "Group" is read-only
volumes/fs/operations/versions/subvolume_base.py: note: In class "SubvolumeBase":
volumes/fs/operations/versions/subvolume_base.py:45: error: Name 'uid' already defined on line 33
volumes/fs/operations/versions/subvolume_base.py:45: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:49: error: Name 'gid' already defined on line 37
volumes/fs/operations/versions/subvolume_base.py:49: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:53: error: Name 'mode' already defined on line 41
volumes/fs/operations/versions/subvolume_base.py:53: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:18:43 +0000 (08:18 -0700)]
mgr/volumes: reverse params passed to `isinstace()`
fixes mypy error:
volumes/fs/operations/clone_index.py: note: In member "track" of class "CloneIndex":
volumes/fs/operations/clone_index.py:38: error: Argument 2 to "isinstance" has incompatible type "Union[VolumeException, Any]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:16:03 +0000 (08:16 -0700)]
mgr/volumes: import VolumeException
fixes mypy errors:
volumes/fs/operations/index.py: note: In member "track" of class "Index":
volumes/fs/operations/index.py:19: error: Name 'VolumeException' is not defined
volumes/fs/operations/index.py: note: In member "untrack" of class "Index":
volumes/fs/operations/index.py:22: error: Name 'VolumeException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:43 +0000 (08:15 -0700)]
mgr/volumes: add `Dict` type
fixes mypy errors:
volumes/fs/operations/op_sm.py:39: error: "object" has no attribute "get"
volumes/fs/operations/op_sm.py:49: error: "object" has no attribute "get"
volumes/fs/operations/lock.py: note: In member "__init__" of class "GlobalLock":
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__exit__"
volumes/fs/operations/lock.py: note: In member "lock_op" of class "GlobalLock":
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__exit__"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:37 +0000 (08:15 -0700)]
mgr/volumes: import errno
fixes mypy errors:
volumes/fs/operations/op_sm.py:36: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:39: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:46: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:49: error: Name 'errno' is not defined
volumes/fs/operations/template.py:5: error: Name 'errno' is not defined
volumes/fs/operations/template.py:14: error: Name 'errno' is not defined
volumes/fs/operations/template.py:23: error: Name 'errno' is not defined
volumes/fs/operations/template.py:32: error: Name 'errno' is not defined
volumes/fs/operations/template.py:42: error: Name 'errno' is not defined
volumes/fs/operations/template.py:45: error: Name 'errno' is not defined
volumes/fs/operations/template.py:62: error: Name 'errno' is not defined
volumes/fs/operations/template.py:74: error: Name 'errno' is not defined
volumes/fs/operations/template.py:85: error: Name 'errno' is not defined
volumes/fs/operations/template.py:94: error: Name 'errno' is not defined
volumes/fs/operations/template.py:103: error: Name 'errno' is not defined
volumes/fs/operations/template.py:112: error: Name 'errno' is not defined
volumes/fs/operations/template.py:121: error: Name 'errno' is not defined
volumes/fs/operations/template.py:130: error: Name 'errno' is not defined
volumes/fs/operations/template.py:139: error: Name 'errno' is not defined
volumes/fs/operations/template.py:148: error: Name 'errno' is not defined
volumes/fs/operations/template.py:158: error: Name 'errno' is not defined
volumes/fs/operations/template.py:169: error: Name 'errno' is not defined
volumes/fs/operations/template.py:180: error: Name 'errno' is not defined
volumes/fs/operations/index.py:18: error: Name 'errno' is not defined
volumes/fs/operations/index.py:21: error: Name 'errno' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
rgw: cls_bucket_list_(un)ordered should clear results collection
Each call to cls_bucket_list_(un)ordered should have an empty
collection to populate with results. Rather than rely on the caller to
insure this, it's more reliable to have these functions do the clear.
Additionally in some cases, a reserve call was added to the collection
to pre-allocate the space needed for the expected number of
results. This will potentially result in fewer re-allocations plus
copies.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Sage Weil [Thu, 5 Mar 2020 14:05:49 +0000 (08:05 -0600)]
Merge PR #33728 into master
* refs/pull/33728/head:
mgr/orch: factor out nice_delta
mgr/orch: show spec age in 'orch ls'
mgr/cephadm: store timestamp with specs
mgr/orch: add created timestamp to ServiceDescription
mgr/orch: include uptime in 'orch ps'
mgr/orch: include AGE column in 'orch ps'
mgr/cephadm: populate new DaemonDescription timestamps
mgr/orch: add new timestamps in DaemonDescription
cephadm: include timestamps for configured, created
cephadm: include timestamps for started, deploy
Reviewed-by: Joshua Schmid <jschmid@suse.de> Reviewed-by: Michael Fritch <mfritch@suse.com>
The ps output names daemons like 'type.foo', e.g., 'mgr.x'. Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.
If a daemon isn't running, we don't know the image_id (hash), so we skip.
But it's also possible to have a running daemon that doesn't report an
image_id... like right after we deploy it when the container hasn't
started up yet. Skip those too.
Kefu Chai [Thu, 5 Mar 2020 05:26:20 +0000 (13:26 +0800)]
Merge pull request #33657 from tchaikov/wip-clang
client,common,mgr,rbd: clang related cleanups
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl> Reviewed-by: Jason Dillaman <dillaman@redhat.com> Reviewed-By: Neha Ojha <nojha@redhat.com> Reviewed-By: Patrick Donelly <pdonell@redhat.com>
Sage Weil [Thu, 5 Mar 2020 01:20:27 +0000 (19:20 -0600)]
Merge PR #33730 into master
* refs/pull/33730/head:
cephadm/vstart-smoke.sh: a smoke test to run against vstart
mgr/cephadm: fix prom reconfig loop when dashboard is disabled
mgr/cephadm: clean up log messages a bit
mgr/cephadm: force remove daemons when removing service
mgr/cephadm: do not error out on service rm|action if no daemons
mgr/orchestrator: allow 'orch rm mon|mgr --force'
Sage Weil [Wed, 4 Mar 2020 23:46:20 +0000 (17:46 -0600)]
mgr/cephadm: do not refresh daemon and device inventory as often
The device inventory rarely changes.
A daemon inventory of 60s may trigger every time around the loop when
doing upgrades, which is pretty slow (esp since it's currently done
synchronously). We're doing a good job (I think?) of queueing refreshes,
so the only reason to time this data out is to catch daemon stops that
are caused by external factors.
Sage Weil [Wed, 4 Mar 2020 20:21:10 +0000 (14:21 -0600)]
mgr/cephadm: fix prom reconfig loop when dashboard is disabled
We need to consider the mgrs a dep even if the prometheus module isn't
enabled, so that we are consistent with _calc_daemon_deps() and don't
reconfig prom in a loop.
Sage Weil [Wed, 4 Mar 2020 20:25:05 +0000 (14:25 -0600)]
Merge PR #33704 into master
* refs/pull/33704/head:
mgr/cephadm: avoid racing updates to spec dict
mgr/cephadm: make _do_upgrade work synchronously
mgr/cephadm/tests/test_cephadm: remove broken rgw_update_fail test
mgr/cephadm: make _apply_service move services
mgr/cephadm: log when we create, destroy daemons
mgr/cephadm: drop async_map_completion decorators from create funcs
mgr/cephadm: _create_osd -> _create_osds
mgr/cephadm: _remove_daemon -> _remove_daemons
Reviewed-by: Joshua Schmid <jschmid@suse.de> Reviewed-by: Michael Fritch <mfritch@suse.com>
Sage Weil [Tue, 3 Mar 2020 18:20:19 +0000 (12:20 -0600)]
mgr/cephadm/tests/test_cephadm: remove broken rgw_update_fail test
This was trying to show that we won't put 2 RGWs from different zones
on the same host. In reality, the test was only passing because it was
specifying an invalid placement spec (count=3 in a 2-host cluster).
It's true that we shouldn't do that because they won't be able to bind,
but the HostAssignment code doesn't handle this case currently.
projects / ceph.git / log