Loic Dachary [Tue, 15 Nov 2016 16:16:37 +0000 (17:16 +0100)]
mon,ceph-disk: add lockbox permissions to bootstrap-osd
ceph-disk --dmcrypt needs to put a config-key and authorize
the OSD to get it back. The corresponding permissions are
added to the bootstrap-osd profile in the monitor.
When preparing the OSD lockbox, use the bootstrap-osd profile instead of
implicitly requiring admin permissions to perform the initial config-key
and auth get-or-create operations.
Ken Dreyer [Mon, 14 Nov 2016 21:49:15 +0000 (14:49 -0700)]
ceph-disk: fix flake8 errors
flake8 3.1.1 surfaces the following issues:
ceph_disk/main.py:173:1: E305 expected 2 blank lines after class or
function definition, found 1
ceph_disk/main.py:5011:1: E305 expected 2 blank lines after class or
function definition, found 1
Mykola Golub [Thu, 10 Nov 2016 13:35:59 +0000 (15:35 +0200)]
librbd: restore journal access when force disabling mirroring
If mirroring is force disabled on a demoted image, the journal was
being left in an inconsistent ownership state.
This is a direct commit to jewel as the fix in the master was
against the newly added async version of mirror disable, which is
not going to be merged to jewel.
Casey Bodley [Wed, 9 Nov 2016 19:27:11 +0000 (14:27 -0500)]
rgw: add missing mutex header for std::once_flag
this fix was added directly to the jewel branch rather than backporting
from master, because the code on master compiles without this specific
include - it's likely included by another header, but backporting would
involve pulling in unrelated changes
Casey Bodley [Fri, 17 Jun 2016 02:51:54 +0000 (22:51 -0400)]
rgw: add pipe fd to set for select() in do_curl_wait()
when HAVE_CURL_MULTI_WAIT is 0, the pipe fd is never added to the
readfds for select(), so FD_ISSET() is always false. this prevents us
from ever trying to read from the fd, and the pipe's buffer eventually
fills up and deadlocks callers of RGWHTTPManager::signal_thread() when
they try to write to the pipe
Brad Hubbard [Fri, 7 Oct 2016 04:51:41 +0000 (14:51 +1000)]
common: Remove the runtime dependency on lsb_release
With modern releases we should be able to make do with the call to
os_release_parse only which uses /etc/os-release which should be available on
most (all?) releases we currently support. this then allows us to remove the
runtime dependency which pulls in several other packages and would be nice to
avoid.
This allows the MDS to respawn using the same executable file even if it
has since been deleted (on Linux). Otherwise, the execv fails because
the readlink returns "/path/to/deleted (deleted)". (There is no path to
the old executable.)
Fixes: http://tracker.ceph.com/issues/17531 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 66a122025f6cf023cf7b2f3d8fbe4964fb7568a7)
Zengran Zhang [Wed, 19 Oct 2016 09:05:27 +0000 (17:05 +0800)]
rgw multisite: fix the increamtal bucket sync init
in the `RGWBucketShardFullSyncCR::operate`, inc_marker will assigned with remote bilog's max_marker.
but the sync_status's inc_marker cant be assigned.so the next step inc sync will always sync
from null log,which means at beginning log.
rgw: get_zonegroup() uses "default" zonegroup if empty
Fixes: http://tracker.ceph.com/issues/17372
An empty zonegroup should be replaced with the "default" zonegroup.
This is needed when dealing with zonegroup set in old bucket info,
that predated setting the buckets' region.
Patrick Donnelly [Mon, 10 Oct 2016 22:16:16 +0000 (18:16 -0400)]
mds: use parse_filesystem in parse_role
This allows us to reuse code in parse_filesystem and avoid
get_filesystem which may fail if the fscid does not exist. This would
result in the program (mon) aborting due to the uncaught exception.
Fixes: http://tracker.ceph.com/issues/17518 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit edc78e46cee356da1e45247c38b7428dd6c965cb)
Yan, Zheng [Sat, 8 Oct 2016 07:16:40 +0000 (15:16 +0800)]
mds: fix false "failing to respond to cache pressure" warning
the false warning happens in following sequence of events
- MDS has cache pressure, sends recall state messages to clients
- Client does not trim as many caps as MDS expected. So MDS
does not reset session->recalled_at
- MDS no longer has cache pressure, it stop sending recall state
messages to clients.
- Client does not release its caps. So session->recalled_at in
MDS keeps unchanged
Yan, Zheng [Sat, 8 Oct 2016 07:16:40 +0000 (15:16 +0800)]
mds: fix false "failing to respond to cache pressure" warning
the false warning happens in following sequence of events
- MDS has cache pressure, sends recall state messages to clients
- Client does not trim as many caps as MDS expected. So MDS
does not reset session->recalled_at
- MDS no longer has cache pressure, it stop sending recall state
messages to clients.
- Client does not release its caps. So session->recalled_at in
MDS keeps unchanged
John Spray [Mon, 19 Sep 2016 14:22:01 +0000 (15:22 +0100)]
mds: use a random nonce in Messenger
The MDS is a client to the OSDs, and responds
to blacklists by respawning itself. Usually
respawns of a daemonized process result in a PID
change, but it's not guaranteed, and it's definitely
not the case when someone runs in foreground (e.g.
teuthology).
Using a random nonce makes sure we won't match
against an existing blacklist entry from a failed
instance of an MDS daemon with the same name as us.
Related to: http://tracker.ceph.com/issues/17236 Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5ba612882750dae6f0b057c660cd283293a18a3f)
Conflicts:
src/ceph_mds.cc : Messenger::create() prototype is different
Sage Weil [Fri, 21 Oct 2016 16:25:08 +0000 (12:25 -0400)]
mon/OSDMonitor: encode OSDMap::Incremental with same features as OSDMap
The Incremental encode stashes encode_features, which is
what we use later to reencode the updated OSDMap. Use
the same features so that the encoding will match!
Conflicts:
src/mon/OSDMonitor.cc: remove references to kraken
if ((osdmap.get_up_osd_features() & CEPH_FEATURE_SERVER_KRAKEN) &&
!osdmap.test_flag(CEPH_OSDMAP_REQUIRE_KRAKEN)) {
string msg = "all OSDs are running kraken or later but the"
" 'require_kraken_osds' osdmap flag is not set";
summary.push_back(make_pair(HEALTH_WARN, msg));
if (detail) {
detail->push_back(make_pair(HEALTH_WARN, msg));
}
} else
Sage Weil [Fri, 30 Sep 2016 22:02:39 +0000 (18:02 -0400)]
mon/OSDMonitor: encode canonical full osdmap based on osdmap flags
If the JEWEL or KRAKEN flags aren't set, encode the full map without
those features. This ensure that older OSDs in the cluster will be able
to correctly encode the full map with a matching CRC. At least, that is
true as long as the encoding changes are guarded by those feature bits.
That appears to be true currently, and we plan to ensure that it is true
in the future as well.
rgw: only enable virtual hosting if hostnames are configured
if no hostnames are configured, all requests were treated as virtual
hosted buckets. require at least one hostname in hostnames_set to
consider setting in_hosted_domain
Robin H. Johnson [Thu, 25 Aug 2016 15:04:34 +0000 (08:04 -0700)]
rgw: Fix Host->bucket fallback logic inversion
The logic (added in 46aae19ee) for falling back to just using the hostname as
the possible bucket name contained an accidental inversion, because
RGWHandler_REST::validate_bucket_name returns success as zero.
Backport: jewel Fixes: http://tracker.ceph.com/issues/17136
Re-Fixes: http://tracker.ceph.com/issues/15975 Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
(cherry picked from commit 70e0289644f4a7205e6c2f75a094ece8ab5ed97c)
David Galloway [Fri, 19 Aug 2016 20:11:32 +0000 (16:11 -0400)]
ceph-post-file: Ignore keys offered by ssh-agent
In my case, I had multiple private keys in ssh-agent which resulted in
the sftp connection failing despite explicitly specifying the private
key to use
Sage Weil [Wed, 2 Nov 2016 13:37:41 +0000 (09:37 -0400)]
ceph-post-file: migrate to RSA SSH keys
DSA keys are being deprecated: http://www.openssh.com/legacy.html
drop.ceph.com will continue to allow the old DSA key but eventually,
users submitting logs using ceph-post-file will run into issues when
OpenSSH completely drops support for the algorithm.
Fixes: http://tracker.ceph.com/issues/14267 Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit ecd02bf3f1c7a07a3271b2736a9e12dd6e897821)
Sage Weil [Sun, 23 Oct 2016 23:40:57 +0000 (18:40 -0500)]
msg: adjust byte_throttler from Message::encode
Normally we never call encode on a message that has a byte_throttler set
because we only use it for messages we received. However, for forwarded
messages that we clear_payload() before resending, we *do* reencode, and in
that case we need to retake the appropriate number of bytes from the
throttler--just like we release them in clear_payload().
Sage Weil [Sat, 22 Oct 2016 18:01:34 +0000 (14:01 -0400)]
messages/MForward: reencode forwarded message if target has differing features
This ensures we reencode the payload with the
appropriate set of features if the client, us, or the
target do not have identical features. Otherwise we
may forward an encoding with more features than the
target can handle.
Sage Weil [Wed, 28 Sep 2016 15:44:28 +0000 (11:44 -0400)]
messages/MForward: fix encoding features
We were encoding the message with the sending client's
features, which makes no sense: we need to encode with
the recipient's features so that it can decode the
message.
The simplest way to fix this is to rip out the bizarre
msg_bl handling code and simply keep a decoded Message
reference, and encode it when we send.
We encode the encapsulated message with the intersection
of the target mon's features and the sending client's
features. This probably doesn't matter, but it's
conceivable that there is some feature-dependent
behavior in the message encode/decode that is important.
Kefu Chai [Fri, 28 Oct 2016 17:54:58 +0000 (01:54 +0800)]
test/ceph_test_msgr: do not use Message::middle for holding transient data
Message::middle is used for holding encoded data, so we we can not stuff
it with payload and leave the "payload" field empty. this change
refactors the ceph_test_msgr by introducing a Payload class which
encodes all test data in it.
Fixes: http://tracker.ceph.com/issues/17728 Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 56896a7ed20869ce91ade4c77c1d6cbab8d50de1)
Conflicts:
src/test/msgr/test_msgr.cc: do not use the new-style DENC()
framework for implementing the encoder of Payload class. DENC() was
introduced after jewel was released.
Kefu Chai [Thu, 13 Oct 2016 05:19:27 +0000 (13:19 +0800)]
mon: send updated monmap to its subscribers
prior to this change, we sends monmap when serving the subscription
requests, but the updates are not sent to the subscribers anymore.
so we need to send latest monmap to subscribes and update the
subscription status accordingly when the monmap is updated.
Patrick Donnelly [Wed, 19 Oct 2016 23:10:14 +0000 (19:10 -0400)]
core: set dumpable flag after setuid
When ceph-* drops drops privileges via setuid, core dumps are no longer
generated because its DUMPABLE flag is cleared. We have to manually
turn that back on.
From prctl(2):
Normally, this flag is set to 1. However, it is reset to the current value contained in the file /proc/sys/fs/suid_dumpable (which by default has the value 0), in the fol‐
lowing circumstances:
* The process's effective user or group ID is changed.
* The process's filesystem user or group ID is changed (see credentials(7)).
* The process executes (execve(2)) a set-user-ID or set-group-ID program, or a program that has capabilities (see capabilities(7)).
Fixes: http://tracker.ceph.com/issues/17650 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ff0e521434acb8c57c65ce3d2a2b9af4cfe914f1)
projects / ceph.git / log