in this change, instead of trying to be smart and to walk all the way
up to the root directory or $top_srcdir, we just check $top_srcdir/.git
directly, as we just know it's there or it does not exist at all.
to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```
clyso [Tue, 31 Mar 2020 12:50:07 +0000 (14:50 +0200)]
doc/mgr/orchestrator: update cephadm shell proposed alias entry
prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.
librbd/cache: undefine dout_subsys before defining it
otherwise we could have following warning from compiler
```
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: warning: "dout_subsys" redefined
13 | #define dout_subsys ceph_subsys_rbd
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:39:
../src/librbd/cache/ReplicatedWriteLog.cc:23: note: this is the location of the previous definition
23 | #define dout_subsys ceph_subsys_rbd_rwl
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:43:
../src/librbd/cache/rwl/ImageCacheState.cc:12: warning: "dout_subsys" redefined
12 | #define dout_subsys ceph_subsys_rbd_rwl
|
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: note: this is the location of the previous definition
13 | #define dout_subsys ceph_subsys_rbd
|
```
because `librbd/cache/rwl/Types.h` includes `libpmemobj.h`, without this
change, `cache/test_mock_ReplicatedWriteLog.cc` will fail to compile as
`libpmemobj.h` might not exist in the default directories compile search
for header files. and hence we could have following failure
```
ceph/src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:11:
86 /root/luyuan/community/github/ceph/src/librbd/cache/rwl/Types.h:8:10: fatal error: libpmemobj.h: No such file or directory
87 8 | #include <libpmemobj.h>
88 | ^~~~~~~~~~~~~~
89 compilation terminated.
90 src/test/librbd/CMakeFiles/unittest_librbd.dir/build.make:1814: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o' failed
91 make[3]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o] Error 1
92 make[3]: *** Waiting for unfinished jobs....
93 CMakeFiles/Makefile2:20379: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/all' failed
94 make[2]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/all] Error 2
95 CMakeFiles/Makefile2:20391: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/rule' failed
96 make[1]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/rule] Error 2
97 Makefile:5375: recipe for target 'unittest_librbd' failed
```
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.
Tim Serong [Thu, 2 Apr 2020 05:46:57 +0000 (16:46 +1100)]
mgr/PyModule: fix missing tracebacks in handle_pyerror()
In certain cases, errors raised in mgr modules don't actually result in a
proper traceback in the mgr log; all you see is a message like "'Hello'
object has no a ttribute 'dneasdfasdf'", but you have no idea where that
came from, which is a complete PITA to debug.
Here's what's going on: handle_pyerror() calls PyErr_Fetch() to get
information about the error that occurred, then passes that information
back to python's traceback.format_exception() function to get the traceback.
If we write code in an mgr module that explicitly raises an exception
(e.g.: 'raise RuntimeError("that didn't work")'), the error value returned
by PyErr_Fetch() is of type RuntimeError, and traceback.format_exception()
does the right thing. If however we accidentally write code that's just
broken (e.g.: 'self.dneasdfasdf += 1'), the error value returned is not
an actual exception, it's just a string. So traceback.format_exception()
freaks out with something like "'str' object has no attribute '__cause__'"
(which we don't actually ever see in the logs), which in turn dumps us in a
"catch (error_already_set const &)" block, which just prints out the
single line error string.
https://docs.python.org/3/c-api/exceptions.html#c.PyErr_NormalizeException
tells us that "Under certain circumstances, the values returned by
PyErr_Fetch() below can be “unnormalized”, meaning that *exc is a class
object but *val is not an instance of the same class.". And that's exactly
the problem we're having here. We're getting a 'str', not an Exception.
Adding a call to PyErr_NormalizeException() turns the value back into a
proper Exception type and traceback.format_exception() now always does the
right thing.
I've also added calls to peek_pyerror() in the catch blocks, so if anything
else ever somehow causes traceback.format_exception to fail, we'll at least
have an idea of what it is in the log.
Fixes: https://tracker.ceph.com/issues/44799 Signed-off-by: Tim Serong <tserong@suse.com>
Sage Weil [Thu, 2 Apr 2020 23:36:39 +0000 (18:36 -0500)]
cephadm: create /var/run/ceph dir via unit.run, not unit file
The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.
Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.
Fixes: https://tracker.ceph.com/issues/44894 Signed-off-by: Sage Weil <sage@redhat.com>
Matthew Oliver [Thu, 5 Mar 2020 02:55:47 +0000 (13:55 +1100)]
cephadm: ceph-iscsi first draft
This if the first draft of the ceph-iscsi in cephadm.
There are a few gotchas when running `rbd-target-api` in a container:
1. We need both the ceph.conf and iscsi-gateway.cfg, so needed to
ability to pass extra config. This latter is based off the spec, so now
the daemon config func api allows you to return a dict of configs:
{ 'config': '<str>' # will be appended to the ceph.conf
'<conf name>': 'str', # Will be dumped in datadir/<conf name>
...
}
It will be up to cephadm to know to bind mount it to the right location.
The first 'config' isn't used by this patch, but makes it possible for
specs or config funcs to append anything? maybe it's overkill.
2. We need the kernel's configfs in the container so we can configure
LIO. There is a chicken and egg problem, configfs isn't mounted on the
host to bind mount when the container starts. So now a check is added to
the `unit.run` and cleanup in the `unit.poststop` scripts for
daemon_type iscsi.
3. rbd-target-api is python and hardcodes a few things, like logging
through `/dev/log` which happens to be a domain socket. So `/dev/log`
also needed to be bind mounted into the continer.
4. The daemon expects the keyring to be in `/etc/ceph` so this needed to
be specifically bind mounted to the correct location too.
As this currently stands this is deploying and starting the api on port
5000, so seems to be "working", also gateway.conf does exist in the
pool. I have yet to set up an iscsi device, but will test that next.
The `rbd-target-api` daemon expects the ssl key and cert to be named a
certain name in the contianer. So SSL isn't working yet. However, I do
hav a PR in ceph-iscsi to look in the mon config-key store for them[0].
Matthew Oliver [Thu, 2 Apr 2020 23:22:02 +0000 (23:22 +0000)]
tox: Fix the tox.ini's to support older versions of tox
The src/cephadm/tox.ini and src/pybind/mgr/tox.ini both don't run
on older versions of tox.
When using tox 2.9.1 both fail for different reasons.
`src/cephadm/tox.ini` fails because `skipsdist=true` only works if it's
directly under the `[tox]` section.
`src/pybind/mgr/tox.ini` fails because older versions of tox can't find
the requirements.txt because they don't like whitespace between the `-r`
and `requirements.txt`.
This patch changes the tox.ini's to be backwards compatible for those
who happen to be running slightly older version of tox.
projects / ceph.git / log