msg/async, v2: fix wrong base for KeepAliveFrameAck.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: frame decoding operates on bufferlist.

This change is driven by buggy buffer's life time management
polluting AuthAuthorizer::bl with dangling raw_static instances.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop ceph_msg_header2 fields duplicating segment info.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop the scaffolding in preamble parsing.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: handle msg authentication failures.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop depedency on uint128_t. Clean up onwire crypto.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: fix cur_msg_size in ::reset_recv_state().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop magic numbers for segments.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: get rid of magic number in SignedEncryptedFrame.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: get rid of the magic number for default alignment.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: decouple onwire segment length from logical length.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: follow the const bl& concept in authenticated_encrypt_update().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop handling of extra segments in ::fill_preamble().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: get rid of magic numbers for alignment.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop reserve() from onwire crypto's TxHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: add con_mode to debug lines

Signed-off-by: Sage Weil <sage@redhat.com>

msg/DispatchQueue: include con_mode in <== line

Signed-off-by: Sage Weil <sage@redhat.com>

common/ceph_strings: get_con_mode_name()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Connection: add get_con_mode()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: clean up preamble comments

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async, v2: improve debug around sending client indent.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: bring back the no-encryption ability.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: workaround con_mode handling.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: drop the throttles bypass.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, v2: READ_MESSAGE_FRONT -> THROTTLE_DONE.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: WaitFrame of V2 can be crypto processed now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: initial multi-segment support for V2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: V2 bypasses throttles just for development.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: rectify reseting security state in ProtocolV2::reset_recv_state().

The reason for that is:

  2019-02-14 19:59:00.774 7f78aebf3700 20 --2- v2:127.0.0.1:40678/0 >>
    v2:127.0.0.1:40680/0 conn(0x5574ae75e480 0x5574ae696000 :-1 s=READY
    pgs=11 cs=0 l=0 rx=0x5574ae2af5f0 tx=0x5574ae79bc00).handle_read_frame_preamble_main
    r=-1
  2019-02-14 19:59:00.774 7f78aebf3700  1 --2- v2:127.0.0.1:40678/0 >>
    v2:127.0.0.1:40680/0 conn(0x5574ae75e480 0x5574ae696000 :-1 s=READY
    pgs=11 cs=0 l=0 rx=0x5574ae2af5f0 tx=0x5574ae79bc00).handle_read_frame_preamble_main
    read frame length and tag failed r=-1 ((1) Operation not permitted)
  2019-02-14 19:59:00.774 7f78aebf3700 10 --2- v2:127.0.0.1:40678/0 >>
    v2:127.0.0.1:40680/0 conn(0x5574ae75e480 0x5574ae696000 :-1 s=READY
    pgs=11 cs=0 l=0 rx=0x5574ae2af5f0 tx=0x5574ae79bc00)._fault

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: switch to CRC32 for V2 preamble blocks.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: bump up preamble block size to 32 bytes.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: get rid of the distiction on main and extra V2 preamble.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: add debug around empty ClientIdent::addrs.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: V2 uses segments instead of next_payload_len, part 1.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: perform V2 frame dispatch in dedicated method.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: implement crc checking for main preamble of V2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: receive V2 messages with new preable format.

Currently we do support messages with single segment only.
Adding multi-segment and proper alignment handling is WIP.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: transmit V2 messages with new preable format.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: reset crypto processors in ProtocolV2::reset_recv_state().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: preamble of V2 Frames is now encrypted and authenticated.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: slightly rework ProtocolV2 preamble crafting.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: reset the rx stream handler in ::handle_read_frame_length_and_tag.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth, msg/async, v2: drop AuthStreamHandler and AES128GCM_StreamHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: Messages in Protocol2 are crypto-processed only once.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: SignedEncryptedFrame uses ceph::crypto::onwire.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: expose message segmentation to ::write_message().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: implement ceph::crypto::onwire with OpenSSL EVP.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: introduce ceph::crypto::onwire interfaces.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: decouple MessageHeaderFrame from SignedEncryptedFrame.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: move Protocol* asserts in SignedEncryptedFrame to compile time.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: simplify encryption handling in the PayloadFrame class.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: drop AES128CBC_HMACSHA256_StreamHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: ensure consistency between con_mode and session_security.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: drop MessageFrame. Use MessageHeaderFrame instead.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: set con_mode and session_security at both peers.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async, auth: switch AuthStreamHandler::rxtx_t to std::unique_ptr.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crypto: AES128GCM_StreamHandler brings authenticated encryption with AES-GCM.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

include: uint128_t -> ceph::uint128_t + using.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: move crypto handling from ProtocolV2 into AuthStreamHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth, msg: dissect AuthStreamHandler from AuthSessionHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth/cephx: make _calc_signature() of CephxSessionHandler private.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: drop {en,de}crypt_message() from AuthSessionHandler.

Yeah, this means wire encryption isn't supposed for V1 anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: introduce DummyAuthSessionHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: make AuthSessionHandler purely abstract.

This is possible because the dropped state is/was required only by:
 * CephxSessionHandler,
 * methods removed by the previous commits.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: drop no_security() from AuthSessionHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: drop get_protocol() and get_key() from AuthSessionHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

auth: drop sign_bufferlist() from AuthSessionHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: drop get_auth_meta() from Protocol.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

msg/async: emphasize ProtocolV2 does authenticated encryption.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #26541 into master

* refs/pull/26541/head:
msg/async: guard protocol Interceptor with preprocessor variable
cmake: add UNIT_TESTS_BUILD preprocessor var based on WITH_TESTS cmake var
test/msgr: remove test_msgr2 unused test
msg/async: msgr2: resolve reconnect races
test/msgr: msgr2 unit tests using the protocol interceptor WIP
msg/async: Protocol interceptor infrastructure

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26458 into master

* refs/pull/26458/head:
MDSMonitor: do not assign standby-replay when degraded

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26468 into master

* refs/pull/26468/head:
qa: config recall settings to test cache drop
qa: check cache dump works without timeout
mds: add 2nd order recall throttle
mds: drive log flush and cache trim during recall
mds: avoid gather assertion when subs exist
mds: output full details for recall threshold

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26422 into master

* refs/pull/26422/head:
mds: start to trim cache when mds enters clientreplay state
mds: avoid evaluating stray dentry multiple times
mds: simplfy stray manager activation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26459 into master

* refs/pull/26459/head:
mds: do not show standby-replay as ranks

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #26486 into master

* refs/pull/26486/head:
mds: remove wrong puts on Message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26237 into master

* refs/pull/26237/head:
Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"
doc/librados: explicitly mention that the C++ API is not stable
ceph.spec: force use of upgrade devtoolset-gcc under RHEL 7
librados: add symbol versioning to the C++ API
librados: add symbol versioning to the C API
librados: revert librados3/libradoscc back to librados2

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async: guard protocol Interceptor with preprocessor variable

Signed-off-by: Ricardo Dias <rdias@suse.com>

cmake: add UNIT_TESTS_BUILD preprocessor var based on WITH_TESTS cmake var

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26395 from ykcowrebbaj/patch-1

doc: corrected options of iscsiadm command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26521 from dillaman/wip-38385

qa/workunits: fixed mon address parsing for rbd-mirror

Merge pull request #26502 from tchaikov/wip-common-coroutine-ifdef

common: define BOOST_COROUTINES_NO_DEPRECATION_WARNING if not yet

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"

This reverts commit 0475dfdea315e5208abc201355523c8dab78469f.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/librados: explicitly mention that the C++ API is not stable

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph.spec: force use of upgrade devtoolset-gcc under RHEL 7

There is a known crash under C++17 mode that requires an updated
devtoolset to address.

See https://bugzilla.redhat.com/show_bug.cgi?id=1616223

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: add symbol versioning to the C++ API

The future goal would be to change the version for each Ceph major
release to ensure C++ applications will need to be recompiled against
the librados C++ API since we don't guarentee ABI stability.

Fixes: https://tracker.ceph.com/issues/38177
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: fix options of iscsiadm command

Signed-off-by: ZhuJieWen <154667715@qq.com>

Merge pull request #26241 from jmolmo/hosts_management_add_rm

mgr/ansible: Add/remove hosts

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

mgr/ansible: Add/remove hosts

- Implementation of operations add/rm hosts
- Improvement of error management in Ansible Runner Service Client

- Addressed @sebastian-philipp suggestions and findings
- Remove the <role> parameter of hosts operations.
Now all the hosts are placed by default in a specific Ansible Inventory group called <orchestrator>.
Discussion on-going about to keep Host management API endpoint or not.
If hosts management is removed, further modifications will be needed to:
- Keep updated the <host inventory> in the orchestrator
- Manage hosts groups when operations are executed
- Adapted new commands (host add/rm) to use to the new <orchestrator_cli> syntax

- Improved error management
- Added return information to Add/remove host methods docstring

- Removed redundant exceptions following @sebastian-philipp suggestion

- Removed duplicated code
- Removed extra space

- Fixed unit tests
- Improved error management in launch playbook

- removed unused param in docstring (orchestrator.py)

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #26525 from liupengs/wip-msg-async-fix-rdma-exchage-port

msg/async/rdma: parse IBSYNMsg.lid as hex when receiving message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26532 from xiexingguo/wip-pool-scrub

mgr: add per pool scrub commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26533 from badone/wip-test-dup_index-trim

test: Verify a log trim trims the dup_index

Reviewed-By: Eric Ivanvich <ivanvich@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26402 from tchaikov/wip-install-deps

 install-deps.sh: selectively install dependencies

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge PR #26551 into master

* refs/pull/26551/head:
osd/PG: fix get debug refcount line

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26511 into master

* refs/pull/26511/head:
mgr: Selftest fails, dashboard/server_port not string

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #26513 into master

* refs/pull/26513/head:
osd/PG: fix comments in merge_from
osd/PG: align past_intervals and last_epoch_clean for fabricated merge target

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26526 into master

* refs/pull/26526/head:
rpm,deb: remove btrfs-tools as a build-dependency
rpm,deb: remove bc as a build-dependency

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #26536 into master

* refs/pull/26536/head:
src/tools: only build dedup tool if WITH_TESTS is set

Reviewed-by: Myoungwon Oh <omwmw@sk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

msg/async/rdma: parse IBSYNMsg.lid as hex when receiving message

function send_msg encode struct IBSYNMsg to a string, and recv_msg parse the string.
In the function send_msg, im.lid (unsigned short int) is formatted into a string with format string %04x,
so, recv_msg should read im.lid from the string with format string %hx instead of %hu (unsigned short).

Fixes: https://tracker.ceph.com/issues/38391
Signed-off-by: Peng Liu <liupeng37@baidu.com>

mgr: add per pool scrub commands

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/PG: fix get debug refcount line

Signed-off-by: Sage Weil <sage@redhat.com>