Sage Weil [Wed, 14 Dec 2016 17:18:29 +0000 (12:18 -0500)]
tasks/workunit: remove kludge to use git.ceph.com
This was hard-coded to ceph.git (almost) and breaks when
you specify --ceph-repo. Remove it entirely. We'll see if
github.com is better at handling our load than it used to
be!
Sage Weil [Thu, 8 Dec 2016 00:25:55 +0000 (18:25 -0600)]
msg/simple/Pipe: avoid returning 0 on poll timeout
If poll times out it will return 0 (no data to read on socket). In 165e5abdbf6311974d4001e43982b83d06f9e0cc we changed tcp_read_wait from
returning -1 to returning -errno, which means we return 0 instead of -1
in this case.
This makes tcp_read() get into an infinite loop by repeatedly trying to
read from the socket and getting EAGAIN.
Fix by explicitly checking for a 0 return from poll(2) and returning
EAGAIN in that case.
Nathan Cutler [Thu, 24 Nov 2016 10:25:35 +0000 (11:25 +0100)]
thrashosds: try ceph-objectstore-tool for 10 minutes
If ceph-objectstore-tool binary is not present, it's likely because we're in
the middle of an upgrade. Do not try to run the binary until we verify that
it's really present. If it is absent, spend up to 10 minutes waiting for it to
appear.
Before this patch there was quite a large window for a race to occur. This
patch doesn't entirely eliminate it, but drastically reduces it.
Samuel Just [Mon, 14 Nov 2016 19:50:23 +0000 (11:50 -0800)]
OSDMonitor: only reject MOSDBoot based on up_from if inst matches
If the osd actually restarts, there is no guarrantee that the epoch will
advance past up_from. If the inst is different, it can't really be a
dup. At worst, it might be a queued MOSDBoot from a previous inst, but
in that case, the real inst would see itself marked up, and then back
down causing it to try booting again.
Loic Dachary [Tue, 29 Nov 2016 08:49:15 +0000 (09:49 +0100)]
upgrade/hammer-x: verify shec before the full upgrade
The hammer-x/stress-split-erasure-code upgrade sequence comes from
hammer-x/stress-split and was modified to fully upgrade the cluster. It
previously upgraded only half of it. Verifying that the shec plugin is
not available and that trying to set it does not crash the OSD or the
MON must be tried before the upgrade is complete.
[Updated write_file to use all feaetures]
[Updated OSDMonitor.cc to use mon->quorum_features instead of the
mon->get_quorum_con_featuers() helper]
[trivial conflict from removed write_file and read_file]
Sage Weil [Mon, 28 Nov 2016 15:29:40 +0000 (10:29 -0500)]
upgrade/hammer-x/parallel: white 'failed to encode'
The problem here has nothing to do with osdmap
encoding, but that hammer -> jewel makes the systemd
transition and installing the package starts
the mons.. before the osds. I'm not sure what
the workaround for that is but the osdmap issue
appears okay, so ignore this for now.
2016-11-18T01:17:08.865 INFO:tasks.ceph.osd.3:Stopping old one...
2016-11-18T01:17:08.865 DEBUG:tasks.ceph.osd.3:waiting for process to exit
2016-11-18T01:17:08.865 INFO:teuthology.orchestra.run:waiting for 300
2016-11-18T01:17:09.199 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
File "/home/teuthworker/src/ceph-qa-suite_wip-whitelist-crc/tasks/ceph_manager.py", line 660, in wrapper
return func(self)
File "/home/teuthworker/src/ceph-qa-suite_wip-whitelist-crc/tasks/ceph_manager.py", line 677, in do_sighup
self.ceph_manager.signal_osd(osd, signal.SIGHUP, silent=True)
File "/home/teuthworker/src/ceph-qa-suite_wip-whitelist-crc/tasks/ceph_manager.py", line 1865, in signal_osd
self.cluster).signal(sig, silent=silent)
File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/daemon.py", line 111, in signal
self.proc.stdin.write(struct.pack('!b', sig))
File "/home/teuthworker/src/teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/file.py", line 377, in write
raise IOError('File is closed')
IOError: File is closed
so trying to avoid this error.
Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
Casey Bodley [Tue, 15 Nov 2016 18:44:27 +0000 (13:44 -0500)]
rgw: start_rgw() polls gateway until it accepts connections
resolves various races between radosgw startup and further operations -
both within the rgw task itself (such as the 'radosgw-admin realm pull'),
and in later tasks
Casey Bodley [Tue, 15 Nov 2016 16:24:25 +0000 (11:24 -0500)]
rgw: add retry/backoff to sync agent requests
resolves an issue where startup of the radosgw-agent races with the
requests we send to it to run sync. uses the requests package with
urllib3 to add retry with backoff to these requests
Sage Weil [Thu, 10 Nov 2016 18:56:24 +0000 (13:56 -0500)]
os/filestore/HashIndex: fix list_by_hash_* termination on reaching end
If we set *next to max, then the caller (a few lines up) doesn't terminate
the loop and will keep trying to list objects in every following hash
dir until it reaches the end of the collection. In fact, if we have an
end bound we will never to an efficient listing unless we hit the max
first.
For one user, this was causing OSD suicides when scrub ran because it
wasn't able to list all objects before the timeout. In general, this would
cause scrub to stall a PG for a long time and slow down requests.
Sage Weil [Fri, 21 Oct 2016 16:25:08 +0000 (12:25 -0400)]
mon/OSDMonitor: encode OSDMap::Incremental with same features as OSDMap
The Incremental encode stashes encode_features, which is
what we use later to reencode the updated OSDMap. Use
the same features so that the encoding will match!
Conflicts:
src/mon/OSDMonitor.cc: remove references to kraken
if ((osdmap.get_up_osd_features() & CEPH_FEATURE_SERVER_KRAKEN) &&
!osdmap.test_flag(CEPH_OSDMAP_REQUIRE_KRAKEN)) {
string msg = "all OSDs are running kraken or later but the"
" 'require_kraken_osds' osdmap flag is not set";
summary.push_back(make_pair(HEALTH_WARN, msg));
if (detail) {
detail->push_back(make_pair(HEALTH_WARN, msg));
}
} else
Sage Weil [Fri, 30 Sep 2016 22:02:39 +0000 (18:02 -0400)]
mon/OSDMonitor: encode canonical full osdmap based on osdmap flags
If the JEWEL or KRAKEN flags aren't set, encode the full map without
those features. This ensure that older OSDs in the cluster will be able
to correctly encode the full map with a matching CRC. At least, that is
true as long as the encoding changes are guarded by those feature bits.
That appears to be true currently, and we plan to ensure that it is true
in the future as well.
David Galloway [Fri, 19 Aug 2016 20:11:32 +0000 (16:11 -0400)]
ceph-post-file: Ignore keys offered by ssh-agent
In my case, I had multiple private keys in ssh-agent which resulted in
the sftp connection failing despite explicitly specifying the private
key to use
Sage Weil [Wed, 2 Nov 2016 13:37:41 +0000 (09:37 -0400)]
ceph-post-file: migrate to RSA SSH keys
DSA keys are being deprecated: http://www.openssh.com/legacy.html
drop.ceph.com will continue to allow the old DSA key but eventually,
users submitting logs using ceph-post-file will run into issues when
OpenSSH completely drops support for the algorithm.
Fixes: http://tracker.ceph.com/issues/14267 Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit ecd02bf3f1c7a07a3271b2736a9e12dd6e897821)
Sage Weil [Sun, 23 Oct 2016 23:40:57 +0000 (18:40 -0500)]
msg: adjust byte_throttler from Message::encode
Normally we never call encode on a message that has a byte_throttler set
because we only use it for messages we received. However, for forwarded
messages that we clear_payload() before resending, we *do* reencode, and in
that case we need to retake the appropriate number of bytes from the
throttler--just like we release them in clear_payload().
Sage Weil [Sat, 22 Oct 2016 18:01:34 +0000 (14:01 -0400)]
messages/MForward: reencode forwarded message if target has differing features
This ensures we reencode the payload with the
appropriate set of features if the client, us, or the
target do not have identical features. Otherwise we
may forward an encoding with more features than the
target can handle.
Sage Weil [Wed, 28 Sep 2016 15:44:28 +0000 (11:44 -0400)]
messages/MForward: fix encoding features
We were encoding the message with the sending client's
features, which makes no sense: we need to encode with
the recipient's features so that it can decode the
message.
The simplest way to fix this is to rip out the bizarre
msg_bl handling code and simply keep a decoded Message
reference, and encode it when we send.
We encode the encapsulated message with the intersection
of the target mon's features and the sending client's
features. This probably doesn't matter, but it's
conceivable that there is some feature-dependent
behavior in the message encode/decode that is important.
Kefu Chai [Fri, 28 Oct 2016 17:54:58 +0000 (01:54 +0800)]
test/ceph_test_msgr: do not use Message::middle for holding transient data
Message::middle is used for holding encoded data, so we we can not stuff
it with payload and leave the "payload" field empty. this change
refactors the ceph_test_msgr by introducing a Payload class which
encodes all test data in it.
Fixes: http://tracker.ceph.com/issues/17728 Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 56896a7ed20869ce91ade4c77c1d6cbab8d50de1)
Conflicts:
src/test/msgr/test_msgr.cc: do not use the new-style DENC()
framework for implementing the encoder of Payload class. DENC() was
introduced after jewel was released.
projects / ceph.git / log