msg/Connection: track peer_id (id portion of entity_name_t) for msgr2

Signed-off-by: Sage Weil <sage@redhat.com>

auth/AuthAuthorizeHandler: add get_supported_methods()

Signed-off-by: Sage Weil <sage@redhat.com>

auth/AuthAuthorizeHandler: fix args for verify_authorizer()

const bufferlists in, pointers for output args.

Signed-off-by: Sage Weil <sage@redhat.com>

auth: constify bufferlist arg to AuthAuthorizer::add_challenge()

Signed-off-by: Sage Weil <sage@redhat.com>

auth/cephx: share all tickets and connection_secret in initial reply

Previously, we would give the client the auth ticket, like a rbd TGT
(ticket granting ticket), and the client would then ask for all of the
other tickets it wants in a separate message.

Instead, have the client specify which tickets it wants up front and pass
them all at the same time.

Also, generate and share the connection_secret, which will be used for
encryption.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async,auth: add AuthConnectionMeta to Protocol

This will hold all of the authentication-related state in an easy-to-find
section that can be accessed via a Connection* or by the protocol stack
(as needed).

Signed-off-by: Sage Weil <sage@redhat.com>

auth/AuthClientHandler: pass in session_key, connection_secret pointers

No functional change.

Signed-off-by: Sage Weil <sage@redhat.com>

auth/AuthServiceHandler: take session_key and connection_secret as args

Allow these methods to populate session and connection secrets.

No functional change (yet).

Signed-off-by: Sage Weil <sage@redhat.com>

auth/cephx: pass more specific type into build_session_auth_info

We were passing CephXServiceTicketInfo in, but the only part of it we
needed was the embedded AuthTicket.  Pass that instead.

No functional change.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Session: separate session creation, peer ident, and registration

- We can now construct a session before we know who it is
- We can later call _ident to identify it
- and also later register it in the session map

Signed-off-by: Sage Weil <sage@redhat.com>

mon/AuthMonitor: bump max_global_id from on_active() and tick()

We should get some runway even if there isn't an auth request coming in.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/AuthMonitor: be more careful with max_global_id

We're potentially handing out ids that haven't committed by increasing
max_global_id before it commits.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: only all ms_handle_authentication() if auth method says we're done

Previously we would call ms_handle_authentication() possibly multiple
times, and without knowning whether it might succeed. Instead, only call
it when start_session() or handle_request() returns >0 to indicate that
we should.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/AuthMonitor: fix "finished with auth" condition check

We are sending the monmap prematurely (before we finish the authentication
handshake).

Signed-off-by: Sage Weil <sage@redhat.com>

auth: clean up AuthServiceHandler::handle_request() args

Signed-off-by: Sage Weil <sage@redhat.com>

auth: clean up AuthServiceHandler::start_session()

- return error code, not type (which never changes)
- take const ref for input args
- pointers for output args

Signed-off-by: Sage Weil <sage@redhat.com>

mon/AuthMonitor: drop unused op arg to assign_global_id()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: separate TAG_AUTH_REQUEST_MORE and TAG_AUTH_REPLY_MORE

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: consolidate authorizer checks

No need to special-case auth methods at this layer.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: move get_auth_allowed into ProtocolV2.cc

We're the only user, and no Dispatchers override.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: trivial cleanup

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_msgr: fix server->client addr discovery

The client's myaddr will be an ANY address, but the internel connection table
will use a v1: or v2: address.  Use the get_peer_addrs() to figure out how to
connect instead.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/{async,simple}: make learned_addr a bit smarter

Only set type ANY if we are a pure client; otherwise, preserve the
type.  Also, only populate the addr if we have a blank ip (sometimes
we already know it from learn_addr_unknowns).

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: very protocol type when looking up existing connections

Since we register client connections as any:, we may have either a ProtocolV1 or V2
connection.  This happens when clients have an imprecise mon search list and connect
to the same mon via both v1 and v2, for example when you do something like

 ceph -m 'v2:127.0.0.1:40648/0,v1:127.0.0.1:40649/0' -s

If we do encounter the other protocol type than what we expect, just mark it down and
proceed.  This is only a temporarily case that happens during mon discovery, the client
is always prepared to retry, and it doesn't actually matter which one succeeds since
it will return a monmap and the client will adapt accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogEntry: use as_legacy_addr()

...just in case addrs has an any addr.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addrvec_t::as_legacy_addr()

Return either the actual legacy addr, or an any addr as a legacy addr.  If
neither is available, lie and return a v2 (or other) addr as a legacy adr.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/AsyncMessenger: drop weird assert

I'm not sure what this was intended for originally...

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: be less verbose

This message has been annoying me

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: learn client addr as type any

Just like AsyncMessenger.  Best to be consistent!

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: set_addr_unknowns(): use front(), not legacy_addr()

We may be passed an any: address.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: drop any: prefix

If it can be any type of address, then simply don't specify the type.  This
is less confusing for humans.

Signed-off-by: Sage Weil <sage@redhat.com>

remove all Messenger::get_myaddr() users

Use get_myaddrs() instead and join the glorious future.

Signed-off-by: Sage Weil <sage@redhat.com>

test/crimson: fix DummyAuthAuthorizer

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: kill get_myaddr()

Not used, and wrong.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV1: use legacy v1 address, not any, in protocol

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Messenger: add get_myaddr_legacy()

This returns a legacy v1 address out of a v1 or any address.  It's
intended to be used in contexts where we *always* want a v1 address,
like SimpleMessenger.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/msg_types: add entity_addr_t::is_any()

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: send entity type and peer_address in Tag::HELLO frame

Signed-off-by: Ricardo Dias <rdias@suse.com>

msg/async: identify client using any: addr

The client can speak v1 or v2, so it is misleading to identify it with a v1 or v2
address (it is either).  This avoid some kludgey workarounds.

We also are a bit more precise about what target_addr means.  It is only used by
the client to indicate which of the peer_addrs we are connecting to, or by a
peer to identify which the peer_addrs we *would* reconnect to.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26198 from Exotelis/mobile-navigation

mgr/dashboard: Add description to menu items on mobile navigation

Merge pull request #25346 from p-na/wip-pna-prettier-modified

mgr/dashboard: Fix modified files only (frontend)

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #26308 from liu-chunmei/ceph_seastar_io_read

test/crimson: fix build failure of test_alien_echo.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26253 from tchaikov/wip-remove-random-cache

common/random_cache: remove unused RandomCache

Reviewed-By: Neha Ojha <nojha@redhat.com>

crimson: fix build error for test_alien_echo.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge pull request #25575 from ThoreKr/doc-pool-stats-master

doc: Add mention of ceph osd pool stats

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26189 from s0nea/wip-dashboard-osd-menu-button

mgr/dashboard: Group buttons together into one menu on OSD page

Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #26296 from ricardoasmarques/fix-iscsi-docs-url

Fix iSCSI docs URL

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26058 from rhcs-dashboard/37917-sso-404-page

mgr/dashboard: SSO - UserDoesNotExist page

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>

Merge pull request #26290 from dzafman/wip-38027-38195

Wip 38027 38195: osd/osd-backfill-space.sh fails

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25918 from rjfd/wip-dashboard-nfs-backend

mgr/dashboard: NFS Ganesha management REST API

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26177 from liewegas/wip-stricter-osd-gate

osd: refuse to start if we're > N+2 from recorded require_osd_release

Reviewed-By: Neha Ojha <nojha@redhat.com>

Merge pull request #26181 from iliul/fix-pybind-mgr-pep8

pybind/mgr: PEP 8 code clean and fix typo

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26258 from liewegas/wip-disable-krb

build: disable kerberos for nautilus

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26259 from rzarzynski/wip-bl-freeze-alloc-unit

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Fix iSCSI docs URL

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #26094 from sebastian-philipp/orchestrator-cli-command

mgr/orchestrator: make use of @CLICommand

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #26178 from dzafman/wip-38070

mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26273 from yehudasa/wip-38174

vstart.sh: fix params generation for monmaptool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: Ignore kill_daemons() error

Workaround for: http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Fix kill_daemon() to check after last large sleep

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Increase timeouts in osd-backfill-space.sh because of failure seen

Fixes: http://tracker.ceph.com/issues/38027
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #26279 from idryomov/wip-krbd-bluestore

qa: objectstore snippets for krbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: ganesha: support no namespace

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25995 from tspmelo/wip-iscsi-ui

mgr/dashboard: iSCSI management UI

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/orchestrator: fix broken `device ls`

* Added `InventoryDevice.from_ceph_volume_inventory`
* Improved documentation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Use scss variable for media queries

Uses the newly created scss breakpoint variables for every media query that was hardcoded so far.

Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: Add description to menu items on mobile navigation

Adds the title of the icons as description right next to it if the resolution
of the browser is below 768px.
Adds a couple of scss variable which define the different breakpoints of
bootstrap. Those variables should be used in media query expressions.

Fixes: https://tracker.ceph.com/issues/37789
Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: SSO - UserDoesNotExist page

* Added sso/404 page for use case when user logs in successfully
  in identity provider but the user does not exist in ceph.

* The page includes a link to perform a logout in IdP
  and return to login page.

Fixes: https://tracker.ceph.com/issues/37917
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard: Add iSCSI Form UI

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add read-only UI for iSCSI

Signed-off-by: Ricardo Marques <rimarques@suse.com>
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: ganesha: notify ganesha daemons to reload configuration

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: ganesha: add LookupError when checking RGW connectivity

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Fix TableComponent when changing columns afterViewInit

Angular was throwing an ExpressionChangedAfterItHasBeenCheckedError if you
changed columns information right after the table was initiated.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Set CdModal max-height

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enhance CdValidators.validateIf method

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add generic Select component

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26202 from smithfarm/wip-fix-s390x-rpm-build

rpm: package crypto libraries for all archs

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/pybind/rados: add binding for rados_notify2

Add a new Ioctx.notify operation that we can call to send a rados
notification over an object.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/orchestrator: make use of @CLICommand

Also Modified some command to allow multiple hosts
Also: Added more tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa: objectstore snippets for krbd

krbd was being tested with filestore, up until recently when the
default for osd_objectstore was changed to bluestore.  This broke
rbd_simple_big.yaml because bluestore_block_size defaults to 10G.
Pick up the sepia setting of 90G from bluestore-bitmap.yaml.

Run fsx subsuite with both filestore and bluestore.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #26106 from yehudasa/wip-rgw-es-fixes

rgw: multiple es related fixes and improvements

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

vstart.sh: fix params generation for monmaptool

Fixes: http://tracker.ceph.com/issues/38174
This broke in the msgr21 case, due to bash globbing in cases where
the generated string somehow matched names of certain files in local
directory. Also need to pass params to the prun function separately,
otherwise everything is quoted together.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #26111 into master

* refs/pull/26111/head:
mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #26123 into master

* refs/pull/26123/head:
mds: refactor MDSContext inheritance

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #26023 from jmolmo/inventory_pb_updated

mgr/ansible: Replace Ansible playbook used to retrieve storage devices data

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #26170 from ifed01/wip-ifed-fix-fsck-read

os/bluestore: deep fsck fails on inspecting very large onodes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #26223 from majianpeng/remove-unused-function

osd: remove unused function.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25278 from ZVampirEM77/wip-deleteobject-policy

rgw: fix obj can still be deleted even if deleteobject policy is set
rgw: cleanup for RGWDeleteObj::verify_permission(

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #26244 from idryomov/wip-krbd-alloc-size

krbd: alloc_size map option and tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rpm: package crypto libraries for all archs

Since 318a8e3c079c937d4e006a9eb4f47c5349648360 we are now building at least the
openssl crypto plugin on all architectures, not just x86_64.

This fixes the non-x86_64 builds for master.

Fixes: 318a8e3c079c937d4e006a9eb4f47c5349648360
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25309 from Devp00l/wip-prometheus-alerting

mgr/dashboard: Prometheus integration

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #26256 into master

* refs/pull/26256/head:
msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

Reviewed-by: Ricardo Dias <rdias@suse.com>

common: make CEPH_BUFFER_ALLOC_UNIT known at compile-time.

It sounds extremely unlikely that any platform Ceph targets can have
PAGE_SIZE lower than 4k. However, to handle these corner cases,
`bufferlist` was rounding to `CEPH_PAGE_SIZE` known at run-time only.
As a result, compilers were enforced to generate pretty inefficient
code like in `ceph::buffer::list::refill_append_space` (self is 1,10%):

```
       │         min(const _Tp& __a, const _Tp& __b)
       │         {
       │           // concept requirements
       │           __glibcxx_function_requires(_LessThanComparableConcept<_Tp>)
       │           //return __b < __a ? __b : __a;
       │           if (__b < __a)
  5,44 │       mov    _DYNAMIC+0x14a0,%rax
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
       │       mov    $0x1000,%ecx
       │     _ZN4ceph6buffer4list19refill_append_spaceEj():
       │         size_t alen = round_up_to(need, CEPH_BUFFER_ALLOC_UNIT) -
       │           sizeof(raw_combined);
       │         buffer::hangable_ptr& new_back = \
       │           hangable_ptr::create(raw_combined::create(alen, 0, get_mempool()));
       │       mov    %rbp,%rdi
       │     _Z11round_up_toImjENSt13make_unsignedINSt11common_typeIJT_T0_EE4typeEE4typeES2_S3_():
  6,21 │       cmpl   $0x1000,(%rax)
  0,78 │       cmovbe (%rax),%ecx
       │       mov    %r12,%rax
       │       xor    %edx,%edx
  0,78 │       mov    %ecx,%ecx
 45,59 │       div    %rcx
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

build: disable kerberos for nautilus

We can re-enable after we branch.  For N, though, there is no point in dragging
in the dependencies and implying there is something that works.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26240 from votdev/improve_get_module_option

mgr: Use Py_BuildValue to create the argument tuple

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26233 from liewegas/wip-bluestore-disable-csum

os/bluestore: add bluestore_ignore_data_csum option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #25077 from rzarzynski/wip-bl-kill_append_buffer

common: drop append_buffer from bufferlist. Use simple carriage instead

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26245 from liewegas/wip-38150

os/bluestore/KernelDevice: use flock(2) for block device lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

msg/async/ProtocolV2: respect policy.resetcheck on cookie==0

It's possible the accepting end knows the cookie but the connecting end
does not if the fault happens before it receives server_ident.  If the
next connection attempt is in the other direction, we need to avoid a
reset or the queued messages at the first site will get lost.

Fixes: http://tracker.ceph.com/issues/38118
Signed-off-by: Sage Weil <sage@redhat.com>

qa: update krbd tests for zeroout

Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>