msg/async/ProtocolV2: store supported features instead of required
We aren't interested in peer_required_features anywhere outside
_handle_peer_banner_payload() -- once we know there is no mismatch,
it's all about peer_supported_features.
Ilya Dryomov [Mon, 11 May 2020 12:06:44 +0000 (14:06 +0200)]
msg/async/ProtocolV2: short circuit empty segments and epilogue
In both msgr2.0 and msgr2.1, segments can be empty. In msgr2.1,
epilogue can be empty as well.
Handle both by calling the respective handler function directly
instead of allocating a buffer::ptr_node for an empty buffer and
passing that through READ[_RXBUF].
msg/async/crypto_onwire: implement msgr2.1 nonce format
Move to a 64-bit counter to avoid wrapping and having to reset
the session before the counter repeats. This is in line with NIST
Recommendation for GCM [1]:
"... this Recommendation suggests, but does not require, that
the leading (i.e., leftmost) 32 bits of the IV hold the fixed
field; and that the trailing (i.e., rightmost) 64 bits hold the
invocation field."
See commit bb61e6a5adc3 ("msg/async/ProtocolV2: avoid AES-GCM nonce
reuse vulnerabilities").
Ilya Dryomov [Mon, 4 May 2020 15:52:13 +0000 (17:52 +0200)]
msg/async/frames_v2: rename and clarify FRAME_FLAGS_LATEABRT
Clarify that the frame can be aborted at any point after the
preamble and the first segment are put on the wire. When that
happens, the remaining segments (including the data segment)
may be filled with zeros.
Ilya Dryomov [Mon, 4 May 2020 15:42:28 +0000 (17:42 +0200)]
msg/async/frames_v2: rename epilogue structs
In preparation for msgr2,1, rename epilogue structs:
epilogue_plain_block_t to epilogue_crc_rev0_block_t and
epilogue_secure_block_t to epilogue_secure_rev0_block_t
(rev0 stands for revision 0).
Also, get rid of size constants that just disguise the
struct type.
Start separating frame assembly and disassembly code from
frame sending, receiving and handling code, so that assembly
and disassembly pieces can be unit tested and hopefully also
shared between different messengers (e.g. crimson).
This commit factors out the assembly code from Frame.
msg/async/crypto_onwire: perform decryption in place
OpenSSL supports in-place decryption so we can avoid
allocating potentially multi-megabyte and strictly aligned
buffer for each decryption operation.
ProtocolV2 actually gets the alignment wrong: after
read_frame_segment() allocates with cur_rx_desc.alignment,
handle_read_frame_segment() effectively replaces that with
segment_t::DEFAULT_ALIGNMENT.
Robin H. Johnson [Fri, 27 Mar 2020 19:48:13 +0000 (20:48 +0100)]
rgw: reject control characters in response-header actions
S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.
Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com> Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com> Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)
Ilya Dryomov [Fri, 6 Mar 2020 19:16:45 +0000 (20:16 +0100)]
msg/async/crypto_onwire: fix endianness of nonce_t
As a AES-GCM IV, nonce_t is implicitly shared between server and
client. Currently, if their endianness doesn't match, they are unable
to communicate in secure mode because each gets its own idea of what
the next nonce should be after the counter is incremented.
Several RFCs state that the nonce counter should be BE, but since we
use LE for everything on-disk and on-wire, make it LE.
The secure mode uses AES-128-GCM with 96-bit nonces consisting of a
32-bit counter followed by a 64-bit salt. The counter is incremented
after processing each frame, the salt is fixed for the duration of
the session. Both are initialized from the session key generated
during session negotiation, so the counter starts with essentially
a random value. It is allowed to wrap, and, after 2**32 frames, it
repeats, resulting in nonce reuse (the actual sequence numbers that
the messenger works with are 64-bit, so the session continues on).
Because of how GCM works, this completely breaks both confidentiality
and integrity aspects of the secure mode. A single nonce reuse reveals
the XOR of two plaintexts and almost completely reveals the subkey
used for producing authentication tags. After a few nonces get used
twice, all confidentiality and integrity goes out the window and the
attacker can potentially encrypt-authenticate plaintext of their
choice.
We can't easily change the nonce format to extend the counter to
64 bits (and possibly XOR it with a longer salt). Instead, just
remember the initial nonce and cut the session before it repeats,
forcing renegotiation.
J. Eric Ivancich [Fri, 20 Mar 2020 15:20:10 +0000 (11:20 -0400)]
rgw: deprecate radosgw-admin orphans sub-commands
The orphans sub-commands have not been maintained for years and it's
unclear whether they're reliable. Additionally, they store logs on the
cluster, so if a user wants to run it to address a nearly full
cluster, it could push the user over the edge. As a result the
(currently experimental) `rgw-orphan-list` tool (separate executable
rather than a sub-command) has been created, which stores intermediate
results on the local filesystem.
Now all three orphans sub-commands produce a warning (on cerr) that
the command is being deprecated. Additionally all three now require
the --yes-i-really-mean-it command-line option to run.
This also documents the "bucket radoslist" sub-command to
`radosgw-admin` in its corresponding man page, a change that was
omitted from the prior commit that added "bucket radoslist"
documentation.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Casey Bodley [Mon, 4 May 2020 20:45:04 +0000 (16:45 -0400)]
vstart: remove commented-out vault config options
we don't have any vstart automation for rgw encryption with vault. if
anyone wants to do local testing, they can follow instructions in
the documentation
Casey Bodley [Mon, 4 May 2020 20:39:59 +0000 (16:39 -0400)]
test/rgw: Cluster.admin() returns output as utf-8 str
fixes the TypeError:
File "rgw_multi/tests.py", line 1107, in test_bucket_sync_disable
check_buckets_sync_status_obj_not_exist(zone, buckets)
File "rgw_multi/tests.py", line 438, in check_buckets_sync_status_obj_not_exist
if log_list.find(':'+bucket+":") >= 0:
TypeError: argument should be integer or bytes-like object, not 'str'
Merge pull request #34773 from ricardoasmarques/cephadm-shell-mount
cephadm: Add shell '--mount' option to mount host file or directory
Reviewed-by: Matthew Oliver <moliver@suse.com> Reviewed-by: Michael Fritch <mfritch@suse.com> Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
cephadm: Add `chown` to unit.run for adoped simple OSDs
Reviewed-by: Jan Fajerski <jfajerski@suse.com> Reviewed-by: Michael Fritch <mfritch@suse.com> Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
qa/rgw: integration test for `rgw-orphan-list` & `radosgw-admin radoslist`
Add teuthology test for `rgw-orphan-list` in a new tool suite under
rgw. It only needs to be tested under one configuration. And the new
tool sub-suite can be used by other tooling int he
future. radosgw-admin `radoslist` is tested indirectly through
`rgw-orphan-list` and therefore does not need its own test.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
J. Eric Ivancich [Tue, 21 Apr 2020 15:28:58 +0000 (15:28 +0000)]
qa/rgw: allow the rgw teuthology task to capture/set dns names
A teuthology workunit might want to use the rgw task, setting the
rgw-dns-name and/or rgw-dns-s3website-name configuration options to
the fully-qualified domain name. Existing code implies that setting
these configuration options to the empty string will do that. However
the current logic does not support that given it has Python
conditionals that treat the empty string as false. This fixes that.
Now the following teuthology tasks YAML will work as expected:
Adds a `radosgw-admin` subcommand and walks the associated bucket
indices and manifests to generate the list of rados objects that
represent the rgw objects in the bucket(s).
Also adds a tool named `rgw-orphan-list`, which uses the radoslist
subcommand, that produces a list in a local file of what appear to be
rgw orphans.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
projects / ceph.git / log