ceph_test_msgr: fix server->client addr discovery

The client's myaddr will be an ANY address, but the internel connection table
will use a v1: or v2: address.  Use the get_peer_addrs() to figure out how to
connect instead.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/{async,simple}: make learned_addr a bit smarter

Only set type ANY if we are a pure client; otherwise, preserve the
type.  Also, only populate the addr if we have a blank ip (sometimes
we already know it from learn_addr_unknowns).

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: very protocol type when looking up existing connections

Since we register client connections as any:, we may have either a ProtocolV1 or V2
connection.  This happens when clients have an imprecise mon search list and connect
to the same mon via both v1 and v2, for example when you do something like

 ceph -m 'v2:127.0.0.1:40648/0,v1:127.0.0.1:40649/0' -s

If we do encounter the other protocol type than what we expect, just mark it down and
proceed.  This is only a temporarily case that happens during mon discovery, the client
is always prepared to retry, and it doesn't actually matter which one succeeds since
it will return a monmap and the client will adapt accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: use as_legacy_addr()

...just in case addrs has an any addr.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::as_legacy_addr()

Return either the actual legacy addr, or an any addr as a legacy addr.  If
neither is available, lie and return a v2 (or other) addr as a legacy adr.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/AsyncMessenger: drop weird assert

I'm not sure what this was intended for originally...

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: be less verbose

This message has been annoying me

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: learn client addr as type any

Just like AsyncMessenger.  Best to be consistent!

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: set_addr_unknowns(): use front(), not legacy_addr()

We may be passed an any: address.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: drop any: prefix

If it can be any type of address, then simply don't specify the type.  This
is less confusing for humans.

Signed-off-by: Sage Weil <sage@redhat.com>

remove all Messenger::get_myaddr() users

Use get_myaddrs() instead and join the glorious future.

Signed-off-by: Sage Weil <sage@redhat.com>

test/crimson: fix DummyAuthAuthorizer

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: kill get_myaddr()

Not used, and wrong.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV1: use legacy v1 address, not any, in protocol

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: add get_myaddr_legacy()

This returns a legacy v1 address out of a v1 or any address.  It's
intended to be used in contexts where we *always* want a v1 address,
like SimpleMessenger.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addr_t::is_any()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: send entity type and peer_address in Tag::HELLO frame

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: identify client using any: addr

The client can speak v1 or v2, so it is misleading to identify it with a v1 or v2
address (it is either).  This avoid some kludgey workarounds.

We also are a bit more precise about what target_addr means.  It is only used by
the client to indicate which of the peer_addrs we are connecting to, or by a
peer to identify which the peer_addrs we *would* reconnect to.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26198 from Exotelis/mobile-navigation

mgr/dashboard: Add description to menu items on mobile navigation

Merge pull request #25346 from p-na/wip-pna-prettier-modified

mgr/dashboard: Fix modified files only (frontend)

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #26308 from liu-chunmei/ceph_seastar_io_read

test/crimson: fix build failure of test_alien_echo.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26253 from tchaikov/wip-remove-random-cache

common/random_cache: remove unused RandomCache

Reviewed-By: Neha Ojha <nojha@redhat.com>

crimson: fix build error for test_alien_echo.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge pull request #25575 from ThoreKr/doc-pool-stats-master

doc: Add mention of ceph osd pool stats

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26189 from s0nea/wip-dashboard-osd-menu-button

mgr/dashboard: Group buttons together into one menu on OSD page

Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #26296 from ricardoasmarques/fix-iscsi-docs-url

Fix iSCSI docs URL

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26058 from rhcs-dashboard/37917-sso-404-page

mgr/dashboard: SSO - UserDoesNotExist page

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>

Merge pull request #26290 from dzafman/wip-38027-38195

Wip 38027 38195: osd/osd-backfill-space.sh fails

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25918 from rjfd/wip-dashboard-nfs-backend

mgr/dashboard: NFS Ganesha management REST API

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26177 from liewegas/wip-stricter-osd-gate

osd: refuse to start if we're > N+2 from recorded require_osd_release

Reviewed-By: Neha Ojha <nojha@redhat.com>

Merge pull request #26181 from iliul/fix-pybind-mgr-pep8

pybind/mgr: PEP 8 code clean and fix typo

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26258 from liewegas/wip-disable-krb

build: disable kerberos for nautilus

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26259 from rzarzynski/wip-bl-freeze-alloc-unit

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Fix iSCSI docs URL

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #26094 from sebastian-philipp/orchestrator-cli-command

mgr/orchestrator: make use of @CLICommand

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #26178 from dzafman/wip-38070

mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26273 from yehudasa/wip-38174

vstart.sh: fix params generation for monmaptool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: Ignore kill_daemons() error

Workaround for: http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Fix kill_daemon() to check after last large sleep

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Increase timeouts in osd-backfill-space.sh because of failure seen

Fixes: http://tracker.ceph.com/issues/38027
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #26279 from idryomov/wip-krbd-bluestore

qa: objectstore snippets for krbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: ganesha: support no namespace

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25995 from tspmelo/wip-iscsi-ui

mgr/dashboard: iSCSI management UI

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/orchestrator: fix broken `device ls`

* Added `InventoryDevice.from_ceph_volume_inventory`
* Improved documentation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Use scss variable for media queries

Uses the newly created scss breakpoint variables for every media query that was hardcoded so far.

Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: Add description to menu items on mobile navigation

Adds the title of the icons as description right next to it if the resolution
of the browser is below 768px.
Adds a couple of scss variable which define the different breakpoints of
bootstrap. Those variables should be used in media query expressions.

Fixes: https://tracker.ceph.com/issues/37789
Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: SSO - UserDoesNotExist page

* Added sso/404 page for use case when user logs in successfully
  in identity provider but the user does not exist in ceph.

* The page includes a link to perform a logout in IdP
  and return to login page.

Fixes: https://tracker.ceph.com/issues/37917
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard: Add iSCSI Form UI

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add read-only UI for iSCSI

Signed-off-by: Ricardo Marques <rimarques@suse.com>
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: ganesha: notify ganesha daemons to reload configuration

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: ganesha: add LookupError when checking RGW connectivity

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Fix TableComponent when changing columns afterViewInit

Angular was throwing an ExpressionChangedAfterItHasBeenCheckedError if you
changed columns information right after the table was initiated.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Set CdModal max-height

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enhance CdValidators.validateIf method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add generic Select component

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26202 from smithfarm/wip-fix-s390x-rpm-build

rpm: package crypto libraries for all archs

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/pybind/rados: add binding for rados_notify2

Add a new Ioctx.notify operation that we can call to send a rados
notification over an object.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/orchestrator: make use of @CLICommand

Also Modified some command to allow multiple hosts
Also: Added more tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa: objectstore snippets for krbd

krbd was being tested with filestore, up until recently when the
default for osd_objectstore was changed to bluestore.  This broke
rbd_simple_big.yaml because bluestore_block_size defaults to 10G.
Pick up the sepia setting of 90G from bluestore-bitmap.yaml.

Run fsx subsuite with both filestore and bluestore.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26106 from yehudasa/wip-rgw-es-fixes

rgw: multiple es related fixes and improvements

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

vstart.sh: fix params generation for monmaptool

Fixes: http://tracker.ceph.com/issues/38174
This broke in the msgr21 case, due to bash globbing in cases where
the generated string somehow matched names of certain files in local
directory. Also need to pass params to the prun function separately,
otherwise everything is quoted together.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #26111 into master

* refs/pull/26111/head:
mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #26123 into master

* refs/pull/26123/head:
mds: refactor MDSContext inheritance

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #26023 from jmolmo/inventory_pb_updated

mgr/ansible: Replace Ansible playbook used to retrieve storage devices data

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #26170 from ifed01/wip-ifed-fix-fsck-read

os/bluestore: deep fsck fails on inspecting very large onodes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #26223 from majianpeng/remove-unused-function

osd: remove unused function.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25278 from ZVampirEM77/wip-deleteobject-policy

rgw: fix obj can still be deleted even if deleteobject policy is set
rgw: cleanup for RGWDeleteObj::verify_permission(

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #26244 from idryomov/wip-krbd-alloc-size

krbd: alloc_size map option and tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rpm: package crypto libraries for all archs

Since 318a8e3c079c937d4e006a9eb4f47c5349648360 we are now building at least the
openssl crypto plugin on all architectures, not just x86_64.

This fixes the non-x86_64 builds for master.

Fixes: 318a8e3c079c937d4e006a9eb4f47c5349648360
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25309 from Devp00l/wip-prometheus-alerting

mgr/dashboard: Prometheus integration

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #26256 into master

* refs/pull/26256/head:
msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

Reviewed-by: Ricardo Dias <rdias@suse.com>

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

It sounds extremely unlikely that any platform Ceph targets can have
PAGE_SIZE lower than 4k. However, to handle these corner cases,
`bufferlist` was rounding to `CEPH_PAGE_SIZE` known at run-time only.
As a result, compilers were enforced to generate pretty inefficient
code like in `ceph::buffer::list::refill_append_space` (self is 1,10%):

```
       │         min(const _Tp& __a, const _Tp& __b)
       │         {
       │           // concept requirements
       │           __glibcxx_function_requires(_LessThanComparableConcept<_Tp>)
       │           //return __b < __a ? __b : __a;
       │           if (__b < __a)
  5,44 │       mov    _DYNAMIC+0x14a0,%rax
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
       │       mov    $0x1000,%ecx
       │     _ZN4ceph6buffer4list19refill_append_spaceEj():
       │         size_t alen = round_up_to(need, CEPH_BUFFER_ALLOC_UNIT) -
       │           sizeof(raw_combined);
       │         buffer::hangable_ptr& new_back = \
       │           hangable_ptr::create(raw_combined::create(alen, 0, get_mempool()));
       │       mov    %rbp,%rdi
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
  6,21 │       cmpl   $0x1000,(%rax)
  0,78 │       cmovbe (%rax),%ecx
       │       mov    %r12,%rax
       │       xor    %edx,%edx
  0,78 │       mov    %ecx,%ecx
 45,59 │       div    %rcx
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

build: disable kerberos for nautilus

We can re-enable after we branch.  For N, though, there is no point in dragging
in the dependencies and implying there is something that works.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26240 from votdev/improve_get_module_option

mgr: Use Py_BuildValue to create the argument tuple

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26233 from liewegas/wip-bluestore-disable-csum

os/bluestore: add bluestore_ignore_data_csum option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #25077 from rzarzynski/wip-bl-kill_append_buffer

common: drop append_buffer from bufferlist. Use simple carriage instead

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26245 from liewegas/wip-38150

os/bluestore/KernelDevice: use flock(2) for block device lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

It's possible the accepting end knows the cookie but the connecting end
does not if the fault happens before it receives server_ident.  If the
next connection attempt is in the other direction, we need to avoid a
reset or the queued messages at the first site will get lost.

Fixes: http://tracker.ceph.com/issues/38118
Signed-off-by: Sage Weil <sage@redhat.com>

qa: update krbd tests for zeroout

Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

common/random_cache: remove unused RandomCache

the only consumer of it was KeyValueStore, which was removed in
19c39faead87655d174965d054de8bbbe4d79067 .

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: krbd discard with alloc_size vs zeroout tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26179 from xiexingguo/wip-37968

crush: fix upmap overkill

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26222 from tchaikov/wip-crimson-heartbeat

crimson/osd: add heartbeat support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: disable GCC-9 warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add heartbeat support

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: move "_preboot" log to where it belongs to

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: bind and listen

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add Messenger::try_bind()

in case the server is just fine with any unused local port number, and it
will report it to monitor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: s/client_msgr/public_msgr/

because this messenger also serves requests from librados clients.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: add pick_address.cc to crimson-common

crimson-osd needs pick_address() for initializing the listening
addresses.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/pick_address: add necessary includes

* add needed forward declaration
* add necessary #includes
* move `#include "common/config.h"` to .cc, as it's not part of
interface.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/pick_address: conditional compile for crimson

crimson can only update configuration async, so we cannot let
pick_addreses() call set_val_or_die() without chaning its function
signature.

for implementing crimson-osd, what we need is only a single
pick_addresses() variant, which returns entity_addrvec_t. so we just
enable it if WITH_SEASTAR is defined instead of adding a wrapper or
reimplement all pick_addresses() variants for crimson-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: add get_fsid() to mon::Client

we need it for creating some messages. `MOSDPing`, for instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: define OSDMapService interface

* define OSDMapService interface to decouple the consumers of
  this interface from its implementation -- the `OSD` class
* implement its methods in `OSD`

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add editor variable settings

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: remove unused method

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: refactor MDSContext inheritance

This commit greatly simplifies things.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26250 from falcon78921/wip-doc-rados-typo

doc: fixed typo in add-or-rm-mons.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: fixed typo in add-or-rm-mons.rst

Redundant use of the word "monitor"

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

common: drop decltype(_buffers) from include/buffers.h.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>