collect the keys instead of filtering a dict,
to address following failure:
```
2020-04-05T12:15:36.275 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-04-05T12:15:36.275 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/cephfs/test_strays.py", line 29, in test_files_throttle
2020-04-05T12:15:36.275 INFO:tasks.cephfs_test_runner: self._test_throttling(self.FILES_THROTTLE)
2020-04-05T12:15:36.276 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/cephfs/test_strays.py", line 96, in _test_throttling
2020-04-05T12:15:36.276 INFO:tasks.cephfs_test_runner: return self._do_test_throttling(throttle_type)
2020-04-05T12:15:36.278 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/cephfs/test_strays.py", line 176, in _do_test_throttling
2020-04-05T12:15:36.278 INFO:tasks.cephfs_test_runner: mds_max_purge_ops = int(self.fs.get_config("mds_max_purge_ops", 'mds'))
2020-04-05T12:15:36.279 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/cephfs/filesystem.py", line 285, in get_config
2020-04-05T12:15:36.279 INFO:tasks.cephfs_test_runner: service_id = random.sample(filter(lambda i: self.mds_daemons[i].running(), self.mds_daemons), 1)[0]
2020-04-05T12:15:36.280 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_py3/virtualenv/lib/python3.5/random.py", line 311, in sample
2020-04-05T12:15:36.280 INFO:tasks.cephfs_test_runner: raise TypeError("Population must be a sequence or set. For dicts, use list(d).")
2020-04-05T12:15:36.280 INFO:tasks.cephfs_test_runner:TypeError: Population must be a sequence or set. For dicts, use list(d).
```
pybind/mgr/dashboard/.pylintrc: drop xrange from the whitelist
since dashboard is now xrange-free, there is no need to have
xrange-builtin in whitelist, moreover, in python3 the error message
cannot be emitted at seeing xrange.
Kefu Chai [Tue, 31 Mar 2020 02:16:40 +0000 (10:16 +0800)]
qa/tasks: use next(iter(..)) for accessing first element in a view
in python2, dict.values() and dict.keys() return lists. but in python3,
they return views, which cannot be indexed directly using an integer index.
there are three use cases when we access these views in python3:
1. get the first element
2. get all the elements and then *might* want to access them by index
3. get the first element assuming there is only a single element in
the view
4. iterate thru the view
in the 1st case, we cannot assume the number of elements, so to be
python3 compatible, we should use `next(iter(a_dict))` instead.
in the 2nd case, in this change, the view is materialized using
`list(a_dict)`.
in the 3rd case, we can just continue using the short hand of
```py
(first_element,) = a_dict.keys()
```
to unpack the view. this works in both python2 and python3.
in the 4th case, the existing code works in both python2 and python3, as
both list and view can be iterated using `iter`, and `len` works as
well.
clyso [Tue, 31 Mar 2020 12:50:07 +0000 (14:50 +0200)]
doc/mgr/orchestrator: update cephadm shell proposed alias entry
prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.
librbd/cache: undefine dout_subsys before defining it
otherwise we could have following warning from compiler
```
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: warning: "dout_subsys" redefined
13 | #define dout_subsys ceph_subsys_rbd
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:39:
../src/librbd/cache/ReplicatedWriteLog.cc:23: note: this is the location of the previous definition
23 | #define dout_subsys ceph_subsys_rbd_rwl
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:43:
../src/librbd/cache/rwl/ImageCacheState.cc:12: warning: "dout_subsys" redefined
12 | #define dout_subsys ceph_subsys_rbd_rwl
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: note: this is the location of the previous definition
13 | #define dout_subsys ceph_subsys_rbd
|
```
because `librbd/cache/rwl/Types.h` includes `libpmemobj.h`, without this
change, `cache/test_mock_ReplicatedWriteLog.cc` will fail to compile as
`libpmemobj.h` might not exist in the default directories compile search
for header files. and hence we could have following failure
```
ceph/src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:11:
86 /root/luyuan/community/github/ceph/src/librbd/cache/rwl/Types.h:8:10: fatal error: libpmemobj.h: No such file or directory
87 8 | #include <libpmemobj.h>
88 | ^~~~~~~~~~~~~~
89 compilation terminated.
90 src/test/librbd/CMakeFiles/unittest_librbd.dir/build.make:1814: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o' failed
91 make[3]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o] Error 1
92 make[3]: *** Waiting for unfinished jobs....
93 CMakeFiles/Makefile2:20379: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/all' failed
94 make[2]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/all] Error 2
95 CMakeFiles/Makefile2:20391: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/rule' failed
96 make[1]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/rule] Error 2
97 Makefile:5375: recipe for target 'unittest_librbd' failed
```
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.
Sage Weil [Thu, 2 Apr 2020 23:36:39 +0000 (18:36 -0500)]
cephadm: create /var/run/ceph dir via unit.run, not unit file
The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.
Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.
Fixes: https://tracker.ceph.com/issues/44894 Signed-off-by: Sage Weil <sage@redhat.com>
Tiago Melo [Tue, 31 Mar 2020 13:55:38 +0000 (13:55 +0000)]
mgr/dashboard: Update npm devDependencies
The following updates required code style changes:
- TSLint updated the logic of ordering imports.
- Prettier improved when to break a command chain into multiple lines.
This commit adds a call to `ceph-facts` role in the first play of this
playbook. This is needed so `ceph-validate` won't fail because of
following error:
zhangdaolong [Tue, 24 Mar 2020 00:51:44 +0000 (08:51 +0800)]
pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output
No lockers are obtained, ImageNotFound exception will be output,
but tht image is always exist.when lockers number is zero,
Should not output any exceptions。
Yingxin Cheng [Sun, 29 Mar 2020 11:17:52 +0000 (19:17 +0800)]
crimson/net: notify if the connection is to be replaced during reset
is_replace=true means the reset connection is going to be replaced by
another accepting connection with the same peer_addr, which currently
only happens under lossy policy when both sides wish to connect to each
other.
Xiubo Li [Tue, 31 Mar 2020 09:09:45 +0000 (05:09 -0400)]
Client: make sure the Finisher's mutex lock not held during it being distructed
The objecter_finisher is already started in Client::Client(), but
in the failure path when initializing and starting the Client object,
we may not get a chance to call the Client::shutdown() to stop the
Finisher thread, which maybe still holding the mutex lock in it. Then
when destrucing the Finisher object the pthread_mutex_destroy() will
fail.
This fix will delay the objecter_finisher thread to start in ::init()
until we're ready to call Client::shutdown on any errors instead.
Fixes: https://tracker.ceph.com/issues/44389 Signed-off-by: Xiubo Li <xiubli@redhat.com>
projects / ceph.git / log