The libradosstriper::RadosStriperImpl::aio_read populates the target
outbl with a static buffer and relies on us reading into it. This was
actually not reliable in the past (it could fail if the rx_buffers
optimization failed due to a retransmit or something else) but nevertheless
libradosstriper requires it to work *at all*.
Resolve this by modifying Objecter to copy the result into any provided
buffer at the lowest layer. This should capture any other such user who
needed this behavior.
On the other hand, it will break any user who inadvertantly reads into a
non-empty bufferlist. Given that any such user would already previously
have seen bad behavior due to the rx_buffers optimization, we expect
there to be 0 such instances.
Sage Weil [Thu, 28 Feb 2019 01:54:16 +0000 (19:54 -0600)]
msg/simple/Pipe: disable rx_buffer code
This reproducibly crashes. Although we don't have a complete understanding
of the exact sequence leading to the crash, we can reproduce, and we have
multiple theoretical cases where it appears to be broken by design.
Fixes: http://tracker.ceph.com/issues/22480 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Wed, 27 Feb 2019 16:25:39 +0000 (10:25 -0600)]
Merge PR #26628 into master
* refs/pull/26628/head:
msg/async: msgr2: expected tags validation
msg/async: msgr2: template specialized write function for frames
msg/async: msgr2: extract frame classes into its own header
Lenz Grimmer [Wed, 27 Feb 2019 12:49:47 +0000 (13:49 +0100)]
doc: Replaced "plugin" with "module" in the Mgr documentation
The documentation currently refers to Ceph Manager Modules as
"plugins" in many places, while the command line interface uses
"module" to enable/disable modules. Replaced all occurences
of "plugin" with "module" in the docs, to avoid confusion and to
be in alignment with the CLI. Also fixed the capitalizations of some
module chapters.
Jason Dillaman [Thu, 14 Feb 2019 14:53:34 +0000 (09:53 -0500)]
librbd: align discard requests to default bluestore allocation size
This will match the new krbd behavior of aligning discards to the
default bluestore minimum allocation size. Requests that are too
small (or the misaligned head / tail of the request) will be
ignored since it will not result in freed space.
Fixes: http://tracker.ceph.com/issues/38146 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jeff Layton [Thu, 21 Feb 2019 22:18:25 +0000 (17:18 -0500)]
mgr/rook: have it create bluestore OSDs in data_directories
When the data_directories field is populated, use that to populate the
spec.storage.nodes.<host>.directories field in the CephCluster object.
That will cue the rook operator to create an OSD in using an area under
that directory for storage.
Jeff Layton [Thu, 21 Feb 2019 22:00:15 +0000 (17:00 -0500)]
mgr/orchestrator: add new list of directories to DriveGroupSpec
Add a new data_directories list to DriveGroupSpec that is intended to
hold a list of directories in which OSDs should be created, as opposed
to the raw devices in data_devices.
Volker Theile [Mon, 25 Feb 2019 14:21:51 +0000 (15:21 +0100)]
mgr/dashboard: Change icon color in notifications
- Use CSS filters to convert the existing icons to white.
- Add new Prometheus SVG image, the previous PNG image has used the color white instead or transparent to fill some areas.
Tim Serong [Mon, 25 Feb 2019 03:47:12 +0000 (14:47 +1100)]
mgr/PyModule: put mgr_module_path first in sys.path
If the various python site packages appear first in sys.path, and there
happens to be a package whose name is the same as an mgr module, mgr will
try to load that thing instead of the expected mgr module. This results
in a very terse couple of errors:
mgr[py] Class not found in module 'deepsea'
mgr[py] Error loading module 'deepsea': (22) Invalid argument
Before this commit, sys.path on my SLE 11 SP1 dev system is:
Sage Weil [Sun, 24 Feb 2019 15:33:32 +0000 (09:33 -0600)]
Merge PR #26466 into master
* refs/pull/26466/head:
msg/async, v2: fix wrong base for KeepAliveFrameAck.
msg/async, v2: frame decoding operates on bufferlist.
msg/async, v2: drop ceph_msg_header2 fields duplicating segment info.
msg/async, v2: drop the scaffolding in preamble parsing.
msg/async, v2: handle msg authentication failures.
msg/async, v2: drop depedency on uint128_t. Clean up onwire crypto.
msg/async, v2: fix cur_msg_size in ::reset_recv_state().
msg/async, v2: drop magic numbers for segments.
msg/async, v2: get rid of magic number in SignedEncryptedFrame.
msg/async, v2: get rid of the magic number for default alignment.
msg/async, v2: decouple onwire segment length from logical length.
msg/async, v2: follow the const bl& concept in authenticated_encrypt_update().
msg/async, v2: drop handling of extra segments in ::fill_preamble().
msg/async, v2: get rid of magic numbers for alignment.
msg/async, v2: drop reserve() from onwire crypto's TxHandler.
msg/async: add con_mode to debug lines
msg/DispatchQueue: include con_mode in <== line
common/ceph_strings: get_con_mode_name()
msg/Connection: add get_con_mode()
msg/async/ProtocolV2: clean up preamble comments
msg/async, v2: improve debug around sending client indent.
msg/async, v2: bring back the no-encryption ability.
msg/async, v2: workaround con_mode handling.
msg/async, v2: drop the throttles bypass.
msg/async, v2: READ_MESSAGE_FRONT -> THROTTLE_DONE.
msg/async: WaitFrame of V2 can be crypto processed now.
msg/async: initial multi-segment support for V2.
msg/async: V2 bypasses throttles just for development.
msg/async: rectify reseting security state in ProtocolV2::reset_recv_state().
msg/async: switch to CRC32 for V2 preamble blocks.
msg/async: bump up preamble block size to 32 bytes.
msg/async: get rid of the distiction on main and extra V2 preamble.
msg/async: add debug around empty ClientIdent::addrs.
msg/async: V2 uses segments instead of next_payload_len, part 1.
msg/async: perform V2 frame dispatch in dedicated method.
msg/async: implement crc checking for main preamble of V2.
msg/async: receive V2 messages with new preable format.
msg/async: transmit V2 messages with new preable format.
msg/async: reset crypto processors in ProtocolV2::reset_recv_state().
msg/async: preamble of V2 Frames is now encrypted and authenticated.
msg/async: slightly rework ProtocolV2 preamble crafting.
msg/async: reset the rx stream handler in ::handle_read_frame_length_and_tag.
auth, msg/async, v2: drop AuthStreamHandler and AES128GCM_StreamHandler.
msg/async: Messages in Protocol2 are crypto-processed only once.
msg/async: SignedEncryptedFrame uses ceph::crypto::onwire.
msg/async: expose message segmentation to ::write_message().
auth: implement ceph::crypto::onwire with OpenSSL EVP.
auth: introduce ceph::crypto::onwire interfaces.
msg/async: decouple MessageHeaderFrame from SignedEncryptedFrame.
msg/async: move Protocol* asserts in SignedEncryptedFrame to compile time.
msg/async: simplify encryption handling in the PayloadFrame class.
auth: drop AES128CBC_HMACSHA256_StreamHandler.
msg/async: ensure consistency between con_mode and session_security.
msg/async: drop MessageFrame. Use MessageHeaderFrame instead.
msg/async: set con_mode and session_security at both peers.
msg/async, auth: switch AuthStreamHandler::rxtx_t to std::unique_ptr.
crypto: AES128GCM_StreamHandler brings authenticated encryption with AES-GCM.
include: uint128_t -> ceph::uint128_t + using.
msg/async: move crypto handling from ProtocolV2 into AuthStreamHandler.
auth, msg: dissect AuthStreamHandler from AuthSessionHandler.
auth/cephx: make _calc_signature() of CephxSessionHandler private.
auth: drop {en,de}crypt_message() from AuthSessionHandler.
auth: introduce DummyAuthSessionHandler.
auth: make AuthSessionHandler purely abstract.
auth: drop no_security() from AuthSessionHandler.
auth: drop get_protocol() and get_key() from AuthSessionHandler.
auth: drop sign_bufferlist() from AuthSessionHandler.
msg/async: drop get_auth_meta() from Protocol.
msg/async: emphasize ProtocolV2 does authenticated encryption.
xie xingguo [Wed, 20 Feb 2019 10:40:02 +0000 (18:40 +0800)]
mgr: add per pool force-recovery/backfill commands
For those with multiple storage pools sharing the same devices,
I think it would make much more sense to offer per-pool
commands to bring pools with high priority, e.g., because they
are hosting data of more importance than others, back to normal
quickly.
Jeff Layton [Fri, 22 Feb 2019 19:40:51 +0000 (14:40 -0500)]
mgr/dashboard: special casing for minikube in run-backend-rook-api-request.sh
For people running minikube for testing, the hostname "minikube" is
often not resolvable. If that's what the hostname is, then get
the address from "minikube ip" instead.
projects / ceph.git / log