Kyr Shatskyy [Sat, 15 Feb 2020 10:12:40 +0000 (11:12 +0100)]
qa/tasks/s3tests: py3 compat
- use string.ascii_uppercase instead string.uppercase
- use six.ensure_str for bytes when required
- use six.ensure_binary if needed
- get rid of dict.itervalues in favor of dict.values
- get rid of cStringIO.StringIO in favor io.BytesIO
Kefu Chai [Fri, 6 Mar 2020 04:17:40 +0000 (12:17 +0800)]
qa/tasks/ceph.py: quote "<kind>" in command line
otherwise bash will intepret "kind" as a file when handling command like
```
sudo zgrep <kind> /var/log/ceph/valgrind/* /dev/null | sort | uniq
```
and try to feed its content to zgrep, and write the output of zgrep
to /var/log/ceph/valgrind/*. this is not the intended behavior. what we
what to do is to pass "<kind>" as an argument to zgrep, along with
the globbed files names which matches "/var/log/ceph/valgrind/*".
volumes/fs/async_cloner.py: note: In function "handle_clone_pending":
volumes/fs/async_cloner.py:71: error: "OpSmException" has no attribute "error"; maybe "errno"?
volumes/fs/async_cloner.py: note: In function "handle_clone_in_progress":
volumes/fs/async_cloner.py:139: error: "OpSmException" has no attribute "error"; maybe "errno"?
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:22:48 +0000 (08:22 -0700)]
mgr/volumes: remove unneeded assignment to `NoneType`
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "get_subvolume_object" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "SubvolumeBase")
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:59 +0000 (08:21 -0700)]
mgr/volumes: add missing OpSmException import
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "upgrade_legacy_subvolume" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:56: error: Name 'OpSmException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:54 +0000 (08:21 -0700)]
mgr/volumes: add missing error code
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "_load_supported_versions" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:35: error: Too few arguments for "VolumeException"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:36 +0000 (08:21 -0700)]
mgr/volumes: fixup format string args
fixes mypy errors:
volumes/fs/purge_queue.py:26: error: Cannot find replacement for positional format specifier 1
volumes/fs/async_cloner.py:37: error: Cannot find replacement for positional format specifier 1
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:30 +0000 (08:21 -0700)]
mgr/volumes: add missing `mgr` param
fixes mypy errors:
volumes/fs/operations/volume.py: note: In function "create_volume":
volumes/fs/operations/volume.py:216: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:223: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:224: error: Too few arguments for "remove_pool"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:25 +0000 (08:21 -0700)]
mgr/volumes: assert self.fs
fixes mypy errors:
volumes/fs/operations/volume.py: note: In member "disconnect" of class "Connection":
volumes/fs/operations/volume.py:94: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
volumes/fs/operations/volume.py:95: error: Item "None" of "Optional[Any]" has no attribute "shutdown"
volumes/fs/operations/volume.py: note: In member "abort" of class "Connection":
volumes/fs/operations/volume.py:105: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:18 +0000 (08:21 -0700)]
mgr/volumes: skip type checking on RTimer class
Unclear why mypy does not like this:
volumes/fs/operations/volume.py: note: In member "run" of class "RTimer":
volumes/fs/operations/volume.py:118: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "interval"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "function"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "args"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "kwargs"
volumes/fs/operations/volume.py:121: error: "RTimer" has no attribute "finished"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:20:01 +0000 (08:20 -0700)]
mgr/volumes: fix positional str formatting
fixes mypy error:
volumes/fs/operations/group.py: note: In function "create_group":
volumes/fs/operations/group.py:135: error: Not all arguments converted during string formatting
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:19:53 +0000 (08:19 -0700)]
mgr/volumes: place getters and setters next to each other
workaround for mypy issue:
https://github.com/python/mypy/issues/1465
fixes mypy errors:
volumes/fs/operations/group.py: note: In class "Group":
volumes/fs/operations/group.py:44: error: Name 'uid' already defined on line 36
volumes/fs/operations/group.py:44: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py:48: error: Name 'gid' already defined on line 40
volumes/fs/operations/group.py:48: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py: note: In function "open_group":
volumes/fs/operations/group.py:170: error: Property "uid" defined in "Group" is read-only
volumes/fs/operations/group.py:171: error: Property "gid" defined in "Group" is read-only
volumes/fs/operations/versions/subvolume_base.py: note: In class "SubvolumeBase":
volumes/fs/operations/versions/subvolume_base.py:45: error: Name 'uid' already defined on line 33
volumes/fs/operations/versions/subvolume_base.py:45: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:49: error: Name 'gid' already defined on line 37
volumes/fs/operations/versions/subvolume_base.py:49: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:53: error: Name 'mode' already defined on line 41
volumes/fs/operations/versions/subvolume_base.py:53: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:18:43 +0000 (08:18 -0700)]
mgr/volumes: reverse params passed to `isinstace()`
fixes mypy error:
volumes/fs/operations/clone_index.py: note: In member "track" of class "CloneIndex":
volumes/fs/operations/clone_index.py:38: error: Argument 2 to "isinstance" has incompatible type "Union[VolumeException, Any]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:16:03 +0000 (08:16 -0700)]
mgr/volumes: import VolumeException
fixes mypy errors:
volumes/fs/operations/index.py: note: In member "track" of class "Index":
volumes/fs/operations/index.py:19: error: Name 'VolumeException' is not defined
volumes/fs/operations/index.py: note: In member "untrack" of class "Index":
volumes/fs/operations/index.py:22: error: Name 'VolumeException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:43 +0000 (08:15 -0700)]
mgr/volumes: add `Dict` type
fixes mypy errors:
volumes/fs/operations/op_sm.py:39: error: "object" has no attribute "get"
volumes/fs/operations/op_sm.py:49: error: "object" has no attribute "get"
volumes/fs/operations/lock.py: note: In member "__init__" of class "GlobalLock":
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__exit__"
volumes/fs/operations/lock.py: note: In member "lock_op" of class "GlobalLock":
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__exit__"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:37 +0000 (08:15 -0700)]
mgr/volumes: import errno
fixes mypy errors:
volumes/fs/operations/op_sm.py:36: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:39: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:46: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:49: error: Name 'errno' is not defined
volumes/fs/operations/template.py:5: error: Name 'errno' is not defined
volumes/fs/operations/template.py:14: error: Name 'errno' is not defined
volumes/fs/operations/template.py:23: error: Name 'errno' is not defined
volumes/fs/operations/template.py:32: error: Name 'errno' is not defined
volumes/fs/operations/template.py:42: error: Name 'errno' is not defined
volumes/fs/operations/template.py:45: error: Name 'errno' is not defined
volumes/fs/operations/template.py:62: error: Name 'errno' is not defined
volumes/fs/operations/template.py:74: error: Name 'errno' is not defined
volumes/fs/operations/template.py:85: error: Name 'errno' is not defined
volumes/fs/operations/template.py:94: error: Name 'errno' is not defined
volumes/fs/operations/template.py:103: error: Name 'errno' is not defined
volumes/fs/operations/template.py:112: error: Name 'errno' is not defined
volumes/fs/operations/template.py:121: error: Name 'errno' is not defined
volumes/fs/operations/template.py:130: error: Name 'errno' is not defined
volumes/fs/operations/template.py:139: error: Name 'errno' is not defined
volumes/fs/operations/template.py:148: error: Name 'errno' is not defined
volumes/fs/operations/template.py:158: error: Name 'errno' is not defined
volumes/fs/operations/template.py:169: error: Name 'errno' is not defined
volumes/fs/operations/template.py:180: error: Name 'errno' is not defined
volumes/fs/operations/index.py:18: error: Name 'errno' is not defined
volumes/fs/operations/index.py:21: error: Name 'errno' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
rgw: cls_bucket_list_(un)ordered should clear results collection
Each call to cls_bucket_list_(un)ordered should have an empty
collection to populate with results. Rather than rely on the caller to
insure this, it's more reliable to have these functions do the clear.
Additionally in some cases, a reserve call was added to the collection
to pre-allocate the space needed for the expected number of
results. This will potentially result in fewer re-allocations plus
copies.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Sage Weil [Thu, 5 Mar 2020 14:05:49 +0000 (08:05 -0600)]
Merge PR #33728 into master
* refs/pull/33728/head:
mgr/orch: factor out nice_delta
mgr/orch: show spec age in 'orch ls'
mgr/cephadm: store timestamp with specs
mgr/orch: add created timestamp to ServiceDescription
mgr/orch: include uptime in 'orch ps'
mgr/orch: include AGE column in 'orch ps'
mgr/cephadm: populate new DaemonDescription timestamps
mgr/orch: add new timestamps in DaemonDescription
cephadm: include timestamps for configured, created
cephadm: include timestamps for started, deploy
Reviewed-by: Joshua Schmid <jschmid@suse.de> Reviewed-by: Michael Fritch <mfritch@suse.com>
The ps output names daemons like 'type.foo', e.g., 'mgr.x'. Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.
If a daemon isn't running, we don't know the image_id (hash), so we skip.
But it's also possible to have a running daemon that doesn't report an
image_id... like right after we deploy it when the container hasn't
started up yet. Skip those too.
Kefu Chai [Thu, 5 Mar 2020 05:26:20 +0000 (13:26 +0800)]
Merge pull request #33657 from tchaikov/wip-clang
client,common,mgr,rbd: clang related cleanups
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl> Reviewed-by: Jason Dillaman <dillaman@redhat.com> Reviewed-By: Neha Ojha <nojha@redhat.com> Reviewed-By: Patrick Donelly <pdonell@redhat.com>
Sage Weil [Thu, 5 Mar 2020 01:20:27 +0000 (19:20 -0600)]
Merge PR #33730 into master
* refs/pull/33730/head:
cephadm/vstart-smoke.sh: a smoke test to run against vstart
mgr/cephadm: fix prom reconfig loop when dashboard is disabled
mgr/cephadm: clean up log messages a bit
mgr/cephadm: force remove daemons when removing service
mgr/cephadm: do not error out on service rm|action if no daemons
mgr/orchestrator: allow 'orch rm mon|mgr --force'
Sage Weil [Wed, 4 Mar 2020 23:46:20 +0000 (17:46 -0600)]
mgr/cephadm: do not refresh daemon and device inventory as often
The device inventory rarely changes.
A daemon inventory of 60s may trigger every time around the loop when
doing upgrades, which is pretty slow (esp since it's currently done
synchronously). We're doing a good job (I think?) of queueing refreshes,
so the only reason to time this data out is to catch daemon stops that
are caused by external factors.
Sage Weil [Wed, 4 Mar 2020 20:21:10 +0000 (14:21 -0600)]
mgr/cephadm: fix prom reconfig loop when dashboard is disabled
We need to consider the mgrs a dep even if the prometheus module isn't
enabled, so that we are consistent with _calc_daemon_deps() and don't
reconfig prom in a loop.
projects / ceph-ci.git / log