msg/async/ProtocolV2: always send non-empty addrvec for self

If we don't know our address yet, send the peer a 0.0.0.0 or :: address with an empty
port and a populated nonce.  That way the peer can infer our final addr the same way
we do from learned_addr.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: never fill out port in myaddr if we didn't bind

If we are a client and didn't bind, then we should not fill in the port for our
address.  The one the peer sent us is just the random port our outgoing connection
happened to land on!

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_msgr: use v2 addrs for simplemessenger

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: don't force write event on every message received

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async/ProtocolV2: be forgiving in server identity check

If we parse something out of mon_host or are given an addr by a user,
we should succeed as long as our target_addr is one of the destination's
addrs.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: fault if we connect to the wrong peer

Someone else may have bound to the same port.  If they identify as someone
else, fault and back off.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION

If we have a connection race, and we lose, we may end up with outgoing
messages *and* be in ACCEPTING_SESSION.  If we then fault, we want to
leave connect_seq at 0 to avoid triggering a reset.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: organizing log messages

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix connection fault when replacing

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix replacing race handling

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix connection race when existing connection is newer

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: assign recv_stamp in handle_message

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix peer_addrs discovery

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: keep authorizer bufferlist across reconnects

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: send keepalive on connection race winner

Msgr2 version of PR #25754

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix client address learning

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix keepalive_ack message

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: do not force updating rotating keys inline

Msgr2 version of PR #25859

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fix mark_down vs accept race

Msgr2 version of PR #25823

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: unregister con from accept vs mark_down race

Msgr2 version of PR #25755

Signed-off-by: Ricardo Dias <rdias@suse.com>

auth/cephx/CephxSessionHandler: use connection_secret for encryption

Signed-off-by: Sage Weil <sage@redhat.com>

msg,cephx: establish a unique connection_secret for every connection

The session_key is unique for each time we newly authenticate with the
mon--e.g., for each client instantiation.  This is not ideal for
encryption, since we don't want to encrypt all connections with the same
key.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: use sha256_digest_t to print signature hex strings

Signed-off-by: Ricardo Dias <rdias@suse.com>

types.h,rgw: merge sha*_digest_t definitions

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: close connection when no authorizer is given

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: formatting fixes

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: send client v2 address when only v1 address is defined

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: add payload length to banner

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: check protocol state after fast dispatch

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: reduce log level for sending messages event

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: store peer entity name in the protocol

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: apply sign/encrypt to messages data payload

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: encryption/decryption of frames

Signed-off-by: Ricardo Dias <rdias@suse.com>

cephx: added encrypt/decrypt bufferlist method to session handler

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: refactored the frame structures

Signed-off-by: Ricardo Dias <rdias@suse.com>

cephx: add sign bufferlist method

Signed-off-by: Ricardo Dias <rdias@suse.com>

options: msgr2 enable/disable signing and encrytion options

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: cephx authentication

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: implement reconnect

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: fault handling

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: messange exchange phase

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: message flow handshake

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: authentication phase

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: exchange peer_type in banner phase

Signed-off-by: Ricardo Dias <rdias@suse.com>

test/msgr: cloned test_msgr test for testing msgr2 protocol

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: msgr2: banner exchange

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: asyncconnection: update the source address info

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: move base class Protocol its own source file

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25991 from jtlayton/wip-dashboard

mgr/dashboard: add a script to run an API request on a rook cluster

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25774 from yehudasa/wip-rgw-tiering-3

rgw: rados tiering

Merge pull request #26073 from rhcs-dashboard/add-tox-ini-command

mgr/dashboard: added command to tox.ini

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #26020 from rhcs-dashboard/issue-37854

mgr/dashboard: Fixed performance details context for host list row selection

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kanika Murarka <kmurarka@redhat.com>

Merge pull request #26007 from rhcs-dashboard/upgrade-python-dev-deps

mgr/dashboard: upgraded python dev dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: added command to tox.ini

* Added 'run' command to run arbitrary commands so
  you don't have to run the whole test/lint suite.
  Use case examples:
  - Run only 1 test:
  tox -e py3-run pytest tests/test_rgw_client.py::RgwClientTest::test_ssl_verify
  - Run pycodestyle for only 1 file:
  tox -e py3-run pycodestyle tests/test_rgw_client.py

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #26032 from jtlayton/wip-rook

mgr/rook: remove dead code and fix bug in url fetching code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #25895 from jan--f/c-v-func-test-local

ceph-volume: functional tests, add libvirt customization

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26044 from tchaikov/wip-build-doc-pyyaml

doc: bump up sphinx and pyyaml versions

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: bump up sphinx and pyyaml versions

* bump up pyyaml version to address CVE-2017-18342, see
  https://nvd.nist.gov/vuln/detail/CVE-2017-18342
* bump up sphinx to the latest stable

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26052 from tchaikov/wip-37976

mgr/zabbix: drop "total_objects" field

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26054 from tchaikov/wip-crimson-dispatcher-fix-ftbfs

crimson/osd: remove "force_new" from ms_get_authorizer()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26045 from hsiang41/Ticket_33736

mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove "force_new" from ms_get_authorizer()

it was removed in e62f45d1, so update crimson-osd accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26046 from liewegas/wip-37976

mgr/telemetry: fix total_objects

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/zabbix: drop "total_objects" field

This field was removed from df output a while back in
342f309

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25712 from familyuu/configuration

mgr/dashboard: Add a 'clear filter' button to configuration page

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Fix devide by zero when total_size is o.

Ref ticket: https://tracker.ceph.com/issues/37736

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>

mgr/telemetry: fix total_objects

This field was removed from df output a while back in
342f309645df886fb96eb401634e38376553e6d9

Fixes: http://tracker.ceph.com/issues/37976
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26031 into master

* refs/pull/26031/head:
mgr/BaseMgrModule: fix mon commands targetting a specific monitor
mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25490 from Aran85/fix-ec-using-cache

osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25920 from wjwithagen/wjw-fix-freebsd-numa

common/numa: Add shim routines for NUMA on FreeBSD.

Merge pull request #25938 from majianpeng/msg-remove-memcopy-message-header

msg/async: reduce additional ceph_msg_header copy.

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26021 from majianpeng/osd-remove-unused-code

osd: remove unused fields

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26040 from tchaikov/wip-blkdev-stat

common/blkdev: check retval of stat()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26041 from tchaikov/wip-kvstore-tool-gcc-ice

tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26012 into master

* refs/pull/26012/head:
qa: add test that down fs does not ERR
mon/MDSMonitor: skip offline ERR for down fs

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge PR #25993 into master

* refs/pull/25993/head:
mds/FSMap: simplify encode/decode
mds: alias reference type for Filesystem
mds: add missing memory header include
include/encoding: eliminate smart_ptr temporaries

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #25976 into master

* refs/pull/25976/head:
Objecter::calc_op_budget: Fix invalid access to extent union member

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #25974 into master

* refs/pull/25974/head:
MDSMonitor: add missing OSDMonitor writeable check
MDSMonitor: codify constraint for writeable OSDMap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #25973 into master

* refs/pull/25973/head:
qa: use simpler fs fail to bring fs down
MDSMonitor: add fs fail command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge PR #25971 into master

* refs/pull/25971/head:
qa: test hex dump_inode
pybind/ceph_argparse: accept int with non-decimal base

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #25650 into master

* refs/pull/25650/head:
client: add ceph_stat/ceph_lstat/ceph_fstat api

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

tool/ceph_kvstore_tool: extract duration calc out

simplify the code a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

to workaround n bug in GCC 7.3, which leads to ICE like

build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.1-2751-gb268822/rpm/el7/BUILD/ceph-14.0.1-2751-gb268822/src/tools/ceph_kvstore_tool.cc:625:1:
internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/BaseMgrModule: fix mon commands targetting a specific monitor

We just need to use the right variant of start_mon_command().

Fixes: http://tracker.ceph.com/issues/37967
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Fixes 61aa7e2e0230bff49e757c32932d1db4bcad5f67

Fixes: http://tracker.ceph.com/issues/37736
Signed-off-by: Sage Weil <sage@redhat.com>

common/blkdev: check retval of stat()

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: use simpler fs fail to bring fs down

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: align head object size with pool alignment

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #25919 from yuriw/wip-yuriw-distros-master

qa/tests: added rhel 7.6

Merge PR #26019 into master

* refs/pull/26019/head:
mon: implement hiding commands in ceph tool

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/rook: wrap url fetch function in try/except block

It's possible for the fetch of this object to fail if the service is
being torn down. Handle that situation gracefully, but log a message.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #25996 from yehudasa/wip-37885

qa/rgw/tempest: either force os_type or select random distro

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #26015 from noahdesu/osd-find-fsid

cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #22797 from dzafman/wip-19753

osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #25132 into master

* refs/pull/25132/head:
os/bluestore: implement kv update on bluefs extents info change.
os/bluestore: use original algorithm to calculate BlueFS gift size for explicit allocation.
os/bluestore: get rid of bluefs space gifting performed in background
os/bluestore: do KV/FM/allocator opening in two stages.
os/bluestore: detach FreelistManager from DB instance.
test/osd/store_test: add test case for explicit bluefs expanding.
os/bluestore: implement explicit bluefs space allocation at slow device
os/bluestore: unify allocator dump on bluefs space failures
os/bluestore: stop keeping bluefs_extents at KV

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/rook: remove init_rook

John evidently added this code early on in development, when we figured
we might be taking over a cluster that is already running under k8s.

I don't see how we'd ever make that work, so just remove this function
for now, and the commented-out callsite.

Reported-by: Sebastian Wagner <sebastian.wagner@suse.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>