Sage Weil [Tue, 3 Mar 2020 21:39:50 +0000 (15:39 -0600)]
mgr/orch: take single placement argument
This is maybe a wash on the 'ceph orch ...' portion of the CLI. However,
it means that elsewhere, like 'ceph fs volume ...', we can be consistent
and have placement be (1) optional and (2) a single arg so that it is
easier to use both positionally and as a flag (--placement=all:true).
Sage Weil [Sat, 7 Mar 2020 03:19:49 +0000 (21:19 -0600)]
Merge PR #33700 into master
* refs/pull/33700/head:
mgr/cephadm: point dashboard at grafana automatically
doc/cephadm/monitoring: document process to set up monitoring with cephadm
Reviewed-by: Alexandra Settle <asettle@suse.com> Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Sage Weil [Fri, 6 Mar 2020 17:26:47 +0000 (11:26 -0600)]
Merge PR #33614 into master
* refs/pull/33614/head:
mgr/cephadm: enable custom TLS certificates for grafana
mgr: enable verification of TLS certs without files
mgr/cephadm: dump config to JSON only once when creating daemons
Kefu Chai [Fri, 6 Mar 2020 04:17:40 +0000 (12:17 +0800)]
qa/tasks/ceph.py: quote "<kind>" in command line
otherwise bash will intepret "kind" as a file when handling command like
```
sudo zgrep <kind> /var/log/ceph/valgrind/* /dev/null | sort | uniq
```
and try to feed its content to zgrep, and write the output of zgrep
to /var/log/ceph/valgrind/*. this is not the intended behavior. what we
what to do is to pass "<kind>" as an argument to zgrep, along with
the globbed files names which matches "/var/log/ceph/valgrind/*".
Sage Weil [Fri, 6 Mar 2020 03:24:53 +0000 (21:24 -0600)]
mgr/cephadm: do not specify --image arg for non-ceph daemons; fix upgrade
If we are calling the cephadm script for a non-ceph daemon (prometheus,
etc), do not specify the --image argument, and do not pull it out of
the config db from sections that don't exist.
Sage Weil [Thu, 5 Mar 2020 16:42:26 +0000 (10:42 -0600)]
mgr/cephadm: make osd create on an existing LV idempotent
If we try to prepare an LV that was already prepared, ceph-volume will
return an error message and code. We want our osd create command to be
idempotent, though, so recognize the error string and continue.
This is an ugly hack, but quicker than changing ceph-volume behavior, and
it is sufficient to stop all of the teuthology failures.
The second part of this is that we have to deploy the daemon on OSDs that
are already prepared and already exist in our osdmap beforehand, but have
never started.
Works-around: https://tracker.ceph.com/issues/44313 Signed-off-by: Sage Weil <sage@redhat.com>
volumes/fs/async_cloner.py: note: In function "handle_clone_pending":
volumes/fs/async_cloner.py:71: error: "OpSmException" has no attribute "error"; maybe "errno"?
volumes/fs/async_cloner.py: note: In function "handle_clone_in_progress":
volumes/fs/async_cloner.py:139: error: "OpSmException" has no attribute "error"; maybe "errno"?
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:22:48 +0000 (08:22 -0700)]
mgr/volumes: remove unneeded assignment to `NoneType`
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "get_subvolume_object" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "SubvolumeBase")
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:59 +0000 (08:21 -0700)]
mgr/volumes: add missing OpSmException import
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "upgrade_legacy_subvolume" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:56: error: Name 'OpSmException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:54 +0000 (08:21 -0700)]
mgr/volumes: add missing error code
fixes mypy error:
volumes/fs/operations/versions/__init__.py: note: In member "_load_supported_versions" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:35: error: Too few arguments for "VolumeException"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:36 +0000 (08:21 -0700)]
mgr/volumes: fixup format string args
fixes mypy errors:
volumes/fs/purge_queue.py:26: error: Cannot find replacement for positional format specifier 1
volumes/fs/async_cloner.py:37: error: Cannot find replacement for positional format specifier 1
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:30 +0000 (08:21 -0700)]
mgr/volumes: add missing `mgr` param
fixes mypy errors:
volumes/fs/operations/volume.py: note: In function "create_volume":
volumes/fs/operations/volume.py:216: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:223: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:224: error: Too few arguments for "remove_pool"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:25 +0000 (08:21 -0700)]
mgr/volumes: assert self.fs
fixes mypy errors:
volumes/fs/operations/volume.py: note: In member "disconnect" of class "Connection":
volumes/fs/operations/volume.py:94: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
volumes/fs/operations/volume.py:95: error: Item "None" of "Optional[Any]" has no attribute "shutdown"
volumes/fs/operations/volume.py: note: In member "abort" of class "Connection":
volumes/fs/operations/volume.py:105: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:21:18 +0000 (08:21 -0700)]
mgr/volumes: skip type checking on RTimer class
Unclear why mypy does not like this:
volumes/fs/operations/volume.py: note: In member "run" of class "RTimer":
volumes/fs/operations/volume.py:118: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "interval"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "function"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "args"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "kwargs"
volumes/fs/operations/volume.py:121: error: "RTimer" has no attribute "finished"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:20:01 +0000 (08:20 -0700)]
mgr/volumes: fix positional str formatting
fixes mypy error:
volumes/fs/operations/group.py: note: In function "create_group":
volumes/fs/operations/group.py:135: error: Not all arguments converted during string formatting
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:19:53 +0000 (08:19 -0700)]
mgr/volumes: place getters and setters next to each other
workaround for mypy issue:
https://github.com/python/mypy/issues/1465
fixes mypy errors:
volumes/fs/operations/group.py: note: In class "Group":
volumes/fs/operations/group.py:44: error: Name 'uid' already defined on line 36
volumes/fs/operations/group.py:44: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py:48: error: Name 'gid' already defined on line 40
volumes/fs/operations/group.py:48: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py: note: In function "open_group":
volumes/fs/operations/group.py:170: error: Property "uid" defined in "Group" is read-only
volumes/fs/operations/group.py:171: error: Property "gid" defined in "Group" is read-only
volumes/fs/operations/versions/subvolume_base.py: note: In class "SubvolumeBase":
volumes/fs/operations/versions/subvolume_base.py:45: error: Name 'uid' already defined on line 33
volumes/fs/operations/versions/subvolume_base.py:45: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:49: error: Name 'gid' already defined on line 37
volumes/fs/operations/versions/subvolume_base.py:49: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:53: error: Name 'mode' already defined on line 41
volumes/fs/operations/versions/subvolume_base.py:53: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:18:43 +0000 (08:18 -0700)]
mgr/volumes: reverse params passed to `isinstace()`
fixes mypy error:
volumes/fs/operations/clone_index.py: note: In member "track" of class "CloneIndex":
volumes/fs/operations/clone_index.py:38: error: Argument 2 to "isinstance" has incompatible type "Union[VolumeException, Any]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:16:03 +0000 (08:16 -0700)]
mgr/volumes: import VolumeException
fixes mypy errors:
volumes/fs/operations/index.py: note: In member "track" of class "Index":
volumes/fs/operations/index.py:19: error: Name 'VolumeException' is not defined
volumes/fs/operations/index.py: note: In member "untrack" of class "Index":
volumes/fs/operations/index.py:22: error: Name 'VolumeException' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:43 +0000 (08:15 -0700)]
mgr/volumes: add `Dict` type
fixes mypy errors:
volumes/fs/operations/op_sm.py:39: error: "object" has no attribute "get"
volumes/fs/operations/op_sm.py:49: error: "object" has no attribute "get"
volumes/fs/operations/lock.py: note: In member "__init__" of class "GlobalLock":
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__exit__"
volumes/fs/operations/lock.py: note: In member "lock_op" of class "GlobalLock":
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__exit__"
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
Michael Fritch [Tue, 3 Mar 2020 15:15:37 +0000 (08:15 -0700)]
mgr/volumes: import errno
fixes mypy errors:
volumes/fs/operations/op_sm.py:36: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:39: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:46: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:49: error: Name 'errno' is not defined
volumes/fs/operations/template.py:5: error: Name 'errno' is not defined
volumes/fs/operations/template.py:14: error: Name 'errno' is not defined
volumes/fs/operations/template.py:23: error: Name 'errno' is not defined
volumes/fs/operations/template.py:32: error: Name 'errno' is not defined
volumes/fs/operations/template.py:42: error: Name 'errno' is not defined
volumes/fs/operations/template.py:45: error: Name 'errno' is not defined
volumes/fs/operations/template.py:62: error: Name 'errno' is not defined
volumes/fs/operations/template.py:74: error: Name 'errno' is not defined
volumes/fs/operations/template.py:85: error: Name 'errno' is not defined
volumes/fs/operations/template.py:94: error: Name 'errno' is not defined
volumes/fs/operations/template.py:103: error: Name 'errno' is not defined
volumes/fs/operations/template.py:112: error: Name 'errno' is not defined
volumes/fs/operations/template.py:121: error: Name 'errno' is not defined
volumes/fs/operations/template.py:130: error: Name 'errno' is not defined
volumes/fs/operations/template.py:139: error: Name 'errno' is not defined
volumes/fs/operations/template.py:148: error: Name 'errno' is not defined
volumes/fs/operations/template.py:158: error: Name 'errno' is not defined
volumes/fs/operations/template.py:169: error: Name 'errno' is not defined
volumes/fs/operations/template.py:180: error: Name 'errno' is not defined
volumes/fs/operations/index.py:18: error: Name 'errno' is not defined
volumes/fs/operations/index.py:21: error: Name 'errno' is not defined
Fixes: https://tracker.ceph.com/issues/44393 Signed-off-by: Michael Fritch <mfritch@suse.com>
rgw: cls_bucket_list_(un)ordered should clear results collection
Each call to cls_bucket_list_(un)ordered should have an empty
collection to populate with results. Rather than rely on the caller to
insure this, it's more reliable to have these functions do the clear.
Additionally in some cases, a reserve call was added to the collection
to pre-allocate the space needed for the expected number of
results. This will potentially result in fewer re-allocations plus
copies.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Casey Bodley [Thu, 5 Mar 2020 16:32:43 +0000 (11:32 -0500)]
rgw: fix string_view formatting in RGWFormatter_Plain
two string_views were being passed directly to vsnprintf where it
expected null-terminated strings. the compiler didn't catch this, and
resulted in segfaults
string_views aren't guaranteed to be null-terminated, so printf formats
have to specify a length as well
Sage Weil [Thu, 5 Mar 2020 14:05:49 +0000 (08:05 -0600)]
Merge PR #33728 into master
* refs/pull/33728/head:
mgr/orch: factor out nice_delta
mgr/orch: show spec age in 'orch ls'
mgr/cephadm: store timestamp with specs
mgr/orch: add created timestamp to ServiceDescription
mgr/orch: include uptime in 'orch ps'
mgr/orch: include AGE column in 'orch ps'
mgr/cephadm: populate new DaemonDescription timestamps
mgr/orch: add new timestamps in DaemonDescription
cephadm: include timestamps for configured, created
cephadm: include timestamps for started, deploy
Reviewed-by: Joshua Schmid <jschmid@suse.de> Reviewed-by: Michael Fritch <mfritch@suse.com>
The ps output names daemons like 'type.foo', e.g., 'mgr.x'. Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.
If a daemon isn't running, we don't know the image_id (hash), so we skip.
But it's also possible to have a running daemon that doesn't report an
image_id... like right after we deploy it when the container hasn't
started up yet. Skip those too.
Kefu Chai [Thu, 5 Mar 2020 05:26:20 +0000 (13:26 +0800)]
Merge pull request #33657 from tchaikov/wip-clang
client,common,mgr,rbd: clang related cleanups
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl> Reviewed-by: Jason Dillaman <dillaman@redhat.com> Reviewed-By: Neha Ojha <nojha@redhat.com> Reviewed-By: Patrick Donelly <pdonell@redhat.com>
projects / ceph.git / log