Jason Dillaman [Tue, 22 Dec 2020 18:36:57 +0000 (13:36 -0500)]
librbd/migration: optionally pull S3 keys from MON config store
This allows the S3 keys to be better protected since the MON can
be configured to restrict access to keys by user and the results
can be encrypted in transit.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 22 Dec 2020 18:34:51 +0000 (13:34 -0500)]
librbd: helper utility to retrieve config from the MON config store
A special "config://" URI prefix can be used to denote configuration
settings that should be (securely) pulled from the MON config store.
This will be first used in a follow-up commit to support storing
the S3 access and secret keys in the MON config store.
This "config://" syntax is already in-use by RGW and ceph-iscsi for
pulling secrets when deployed via cephadm.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 22 Dec 2020 14:46:04 +0000 (09:46 -0500)]
librbd/migration: require snapshot when importing from native source
Since we cannot mark the source image read-only when running in import-only
migration mode, we should require the user to provide a snapshot to ensure
that data cannot change while the migration is running.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Xuehan Xu [Tue, 29 Dec 2020 04:06:43 +0000 (12:06 +0800)]
crimson/osd: do not splice osd_op indata
As each osd_op's indata holds only its own data, there's no need to drop their it.
On the other hand, indata is still needed when osd_op needs to be redone in case of
acting set change.
Avan Thakkar [Thu, 17 Dec 2020 09:44:46 +0000 (15:14 +0530)]
mgr/dashboard: Fix for copy2Clipboard failing to copy on the first time.
Fixes: https://tracker.ceph.com/issues/48601 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Fix for the copy2Clipboard button to work on the first click itself.
Compile error is encountered when HAVE_LEVELDB_FILTER_POLICY is not
defined.
/ceph/src/kv/LevelDBStore.cc: In member function 'int LevelDBStore::load_leveldb_options(bool, leveldb::Options&)':
/ceph/src/kv/LevelDBStore.cc:96:22: error: invalid initialization of reference of type 'const string& {aka const std::__cxx11::basic_string<char>&}' from expression of type 'bool'ceph_abort_msg(0 == "bloom size set but installed leveldb doesn't support bloom filters");
~~^~~~
/ceph/src/include/ceph_assert.h:90:69: note: in definition of macro 'ceph_abort_msg'
::ceph::__ceph_abort( __FILE__, __LINE__, __CEPH_ASSERT_FUNCTION, msg)
^~~
/ceph/src/include/ceph_assert.h:62:19: note: in passing argument 4 of 'void ceph::__ceph_abort(const char*, int, const char*, const string&)'
[[noreturn]] void __ceph_abort(const char *file, int line, const char *func,
^~~~~~~~~~~~
src/kv/CMakeFiles/kv.dir/build.make:130: recipe for target 'src/kv/CMakeFiles/kv.dir/LevelDBStore.cc.o' failed
Signed-off-by: Jerry Lee <leisurelysw24@gmail.com>
Patrick Donnelly [Thu, 24 Dec 2020 04:20:09 +0000 (20:20 -0800)]
Merge PR #38668 into master
* refs/pull/38668/head:
osdc: Fix return type of map waiting functions
monc: Use post rather than defer
common/async: Hold lock in constructor of blocked_result
osdc: Don't keep wrapping completion in wait_for_latest_osdmap
neorados: Replace unsafe uses of dispatch with post
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Wed, 23 Dec 2020 09:21:35 +0000 (17:21 +0800)]
test/crimson: add abort timer to messenger test
Use a 120s timer to force abort the messenger test which should normally
succeeds in 60s, in order to:
* Fail the make check to print logs about where it is blocking;
* Free the listening IP address for the next make check;
Patrick Donnelly [Tue, 22 Dec 2020 20:58:11 +0000 (12:58 -0800)]
Merge PR #37721 into master
* refs/pull/37721/head:
test: add tests for new snapshot APIs
pybind/cephfs: python bindings for new snapshot APIs
client: Snapshot cephfs APIs
mds: include snapshot metadata in trace reply to client
mds: include snap metadata on-disk on snapshot creation
mds: include metadata in SnapInfo structure
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Zac Dover [Thu, 19 Nov 2020 14:18:51 +0000 (00:18 +1000)]
doc/dev: update unit-tests tox section
This commit
- alters the structure of the
unit-test section of the Developer Guide
that discusses the tox tool, which is
used to run certain unit tests
- corrects some grammar
- updates the list of locations of tox.ini
files
- This also incorporates N. Cutler's suggestions.
Kefu Chai [Mon, 21 Dec 2020 17:07:37 +0000 (01:07 +0800)]
include/denc: use pair<const K,V> in range-based for loop
map<K,V>::value_type is pair<const K, V>, so if we use range-based for
loop when iterating through a map, we should use pair<const K,V> instead
of pair<K,V>, the latter also compiles, but it might create a temporary
object of pair<K,V> from pair<const K,V>. GCC-11 complains at seeing
this:
../src/include/denc.h:1002:21: warning: loop variable ‘e’ of type ‘const T&’ {aka ‘const std::pair<OSDPerfMetricQuery, OSDPerfMetricReport>&’} binds to a tem\
porary constructed from type ‘const std::pair<const OSDPerfMetricQuery, OSDPerfMetricReport>’ [-Wrange-loop-constru
ct]
1002 | for (const T& e : s) {
| ^
this change
* use the value_type of container in `maplike_details<Container>`,
so we can avoid the overhead of creating temporay objects when
encoding a map
* define denc_traits for std::pair<const A, B> as well, so the elements
of a map can be encoded using denc facility
Cory Snyder [Mon, 21 Dec 2020 14:33:22 +0000 (09:33 -0500)]
mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock
A thread that holds the GIL while attempting to acquire a mutex will cause a deadlock
if another thread owns the mutex and is waiting on the GIL. The GIL must not be treated
like an ordinary mutex since it may be preempted at any time or released when doing
blocking I/O. Such deadlocks are severe since they starve all threads from access to the
GIL and therefore prevent any Python code from running until the mgr process is restarted.
Kefu Chai [Sun, 20 Dec 2020 05:16:17 +0000 (13:16 +0800)]
cmake: install python scripts into /usr/sbin even if DESTDIR is empty
it's reported that on Alphine, "make install" installs
`ceph-volume-systemd` into `/usr/bin` instead of `/usr/sbin`, and this
breaks `systemd/ceph-volume@.service`. so in this change, we always
install this script into `/usr/sbin`, even if `DESTDIR` is not defined.
Kefu Chai [Sun, 20 Dec 2020 05:10:16 +0000 (13:10 +0800)]
pybind/ceph_argparse.py: use a safe value for timeout
we have reports that on arm32 machines, it timed out immediately, so
to prevent it from int overflow, use a safer value instead of
(1 << (32 - 1)) - 1.
Kefu Chai [Sun, 20 Dec 2020 05:02:46 +0000 (13:02 +0800)]
erasure-code: bail out early if chunks to be decoded is empty
otherwise we will read from uninitialized memory for blocksize.
update tests accordingly, as the number of decoded chunks is 0 if
* the number of available chunks is 0 and
* the number of requested chunks is not 0
after this change, because, in that case, the decode fails before any
chunk is allocated. but otherwise, the output chunks are still
allocated. so instead checking the number of output chunks, the test is
removed.
Kefu Chai [Sun, 20 Dec 2020 03:39:49 +0000 (11:39 +0800)]
erasure-code: print warnings using dout(10)
stderr is closed once the process is daemonized, while `derr` is for
fatal error or logic error in program. dout(10) is more appropriate in
this case, as it is an expected error from callee's perspective.
projects / ceph.git / log