common: drop the unused reset_comparator() from SharedLRU.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop the unused get_comparator() from SharedLRU.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26188 from Exotelis/tracker-37916

mgr/dashboard: Cleanup of the cluster and audit log

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #25542 from jan--f/c-v-extend-batch

Extend batch to accept explicit device lists

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26205 from cbodley/wip-38115

qa/mon: fix cluster support for monmap bootstrap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26204 from pritha-srivastava/wip-rgw-signature-mismatch

rgw: Fix for SignatureMismatchError in s3 commands.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25403 from liyichao/rdigest

tools: Add clear-data-digest command to objectstore tool.

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #26158 from dzafman/wip-38053

Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/mon: fix cluster support for monmap bootstrap

-filter out mons from other clusters
-fix parsing of mon name from role

Fixes: http://tracker.ceph.com/issues/38115
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: Fix for SignatureMismatchError in s3 commands.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #25638 from ricardoasmarques/iscsi-management-w

mgr/dashboard: iSCSI management API

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26172 from rhcs-dashboard/fix-skipped-api-tests

mgr/dashboard: fix skipped backend API tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph-volume: don't crash on unusable data devices

Also add test case for a MixedStrategy with unusable data devices.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: add a wal device specific unit test and size validator

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: add osd_ids argument

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #26091 from tchaikov/wip-36737

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

Tested-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #26142 into master

* refs/pull/26142/head:
doc/dev/cephx: document current implementation of cephx

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #26097 from trociny/wip-rbd-mirror-perf-lock

rbd-mirror: guard access to image replayer perf counters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26168 from shyukri/qa-rbd-replace-bench-write

qa: rbd/workunits : Replace "rbd bench-write" with "rbd bench --io-type write"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: fix skipped backend API tests

* When the creation of the cluster is delegated to vstart_runner.py
(--create or --create-target-only) the amount of MGRs required
is calculated by the script so there is no more skipped tests
due to insufficient amount of MGRs.

* Additionally, this issue is not reproducible anymore:
Fixes: https://tracker.ceph.com/issues/37964
* Fixed typo: TEUTHOLOFY_PY_REQS

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #26162 from familyuu/crush

mgr/dashboard: CRUSH map viewer RFE

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #26182 from p-na/wip-pna-fix-pool-reload

mgr/dashboard: Fix reloading of pool listing

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #26143 from LenzGr/master-documentation

doc: Updated feature list and overview in dashboard.rst

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #25893 from sebastian-philipp/orchestrator-current-status

doc/orchestrator: Aligned Documentation with specification

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

mgr/dashboard: CRUSH map viewer RFE

Fixes: http://tracker.ceph.com/issues/37794
Signed-off-by: familyuu <guodan1@lenovo.com>

Merge pull request #26175 from neha-ojha/wip-37919

osd/PG.cc: account for missing set irrespective of last_complete

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #25737 from arthurhsliu/patch-1

doc: fix typo in erasure coding example

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/dev/cephx: document current implementation of cephx

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26157 from cbodley/wip-rgw-olh-repair

rgw: repair olh attributes that were broken by sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #26002 from pritha-srivastava/wip-sts-assumerole-webidentity

rgw: Initial commit for AssumeRoleWithWebIdentity.

Merge pull request #26110 from yaozongyou/fix-statistics-bug

mon: fix pg_sum_old not copied correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/orchestrator: Aligned Documentaiton with specification

Also: add current implementation status
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Cleanup cluster and audit log

Applies the following changes:
- Makes the timestamp bold and not the message
- Colors the priority according to its level
- Indents the message correct if it's longer than one line
- Displays a message when the log is empty

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: Refactor log pipe

The pipe returns a class name instead of an object now. This has the advantage,
that the layout can be modified directly in scss and keeps the code seperated
from the layout.

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>

mgr/dashboard: Move and rename pipe

Moves file to the shared directory to make it available for every module.
Also renames the file, because it will be refactored.

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>

Merge pull request #26090 from tchaikov/wip-crimson-osdmap

crimson: persist/load osdmap to/from store

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Fix reloading of pool listing

Remove broken functionality that prevents pools from being reloaded after
deletion. The code also introduced a different problem; when clicking on a tab
shortly after the page has been loaded, the code will restore the tab to the
wrong one.

Introduced by 9e913bfd59c3a08c9e06bd8535bb7b92593ba549

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #26135 from Tinatinak/fix_permission_decorator

mgr/dashboard: Correct permission decorator

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26165 from rhcs-dashboard/ng-envbuild-e2e

mgr/dashboard: added 'env_build' to 'npm run e2e'

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

crimson/osd: fix boot process

* send my client addresses to mon
* always update osdmap with the latest one

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: load_pgs()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: add omap_get_values()

osd uses it for storing the pg_info.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/HitSet: mark copy ctor of HitSet::Params noexcept

to be returned using seastar::future<...> the value type should satisfy
std::is_nothrow_constructible<T>.

with this change, pg_pool_t will be nothrow_constructible. and hence
can be returned using seastar::future<pg_pool_t>. otherwise
std::is_nothrow_constructible<pg_pool_t>::value would be false.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add operator<< to OSDState

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: note missing coll/oid in exception

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: load/store osdmap from/to store

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26133 from dillaman/wip-37913

rbd: implement new 'rbd perf image iostat/iotop' commands

Reviewed-by: Mykola Golub <mgolub@suse.com>

crimson/osd: add "--mkfs" support to crimson-osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: move more features from MemStore to CyanStore

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Correct permission decorator

'CreatePermission' is required in order to restore an image from trash.

Signed-off-by: Tina Kallio <tina.kallio@gmail.com>

rgw: Initial commit for AssumeRoleWithWebIdentity.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rbd: new 'perf image iostat/iotop' actions

This provides an 'iostat' and 'top'-like IO monitor for all
RBD images

Fixes: http://tracker.ceph.com/issues/37913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: add RGWRados::repair_olh() to reconstruct olh attrs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWRados::bi_get_olh() to read/decode olh entry

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: bi_get() takes bucket_info

and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add BucketShard::init overload for bucket_info,obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: simplify pool namespace positional argument handling

The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/rbd_support: support module for RBD features that require coordination

This initially adds mgr commands to collect RBD performance metrics
from the OSD dynamic perf collector:

  rbd perf image stats: collects IO rate statistics
  rbd perf image counters: collects raw IO counters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25112 from dzafman/wip-scrub-warning

scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26081 from cbodley/wip-38007

rgw: bucket full sync handles delete markers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #25137 from yehudasa/wip-archive-zone-3

rgw: archive zone

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

ceph-volume: add explicit mixed-type functional tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

rgw: bucket full sync handles delete markers

full sync uses CLS_RGW_OP_LINK_OLH_DM for delete markers instead of
trying to fetch them from the source zone

Fixes: http://tracker.ceph.com/issues/38007
Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add multisite test_suspended_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add multisite test_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: use version_id from key directly

now that PutObj returns the x-amz-version-id, it's available through
Key.version_id without resorting to Bucket.list_versions()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: check_bucket_eq() supports delete markers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: check_bucket_eq() uses bucket.list_versions()

get_all_versions() is documented as a lower-level api that doesn't
handle paging, ands suggests list_versions() instead. also caches the
results to avoid listing each bucket twice

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25794 from cbodley/wip-rgw-sync-olh-attrs

rgw: fetch_remote_obj filters out olh attrs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

qa:rbd/workunits : Replace `rbd bench-write` with `rbd bench --io-type write`

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>

Merge PR #24546 into master

* refs/pull/24546/head:
msg/async/ProtocolV2: clear dispatch throttle on connection stop
msg/async: fix should_use_msgr2 behavior (including monc)
msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
ceph_test_msgr: do not connect_to on the client side
msg/async: do not connect from server
msg/async: do not use peer to addr detection; use getsockname()
msg/async/ProtocolV2: always send non-empty addrvec for self
msg/async: never fill out port in myaddr if we didn't bind
ceph_test_msgr: use v2 addrs for simplemessenger
msg/async: msgr2: don't force write event on every message received
msg/async/ProtocolV2: be forgiving in server identity check
msg/async/ProtocolV2: fault if we connect to the wrong peer
msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION
msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION
msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection
msg/async: msgr2: organizing log messages
msg/async: msgr2: fix connection fault when replacing
msg/async: msgr2: fix replacing race handling
msg/async: msgr2: fix connection race when existing connection is newer
msg/async: msgr2: assign recv_stamp in handle_message
msg/async: msgr2: fix peer_addrs discovery
msg/async: msgr2: keep authorizer bufferlist across reconnects
msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds
msg/async: msgr2: send keepalive on connection race winner
msg/async: msgr2: fix client address learning
msg/async: msgr2: fix keepalive_ack message
msg/async: msgr2: do not force updating rotating keys inline
msg/async: msgr2: fix mark_down vs accept race
msg/async: msgr2: unregister con from accept vs mark_down race
auth/cephx/CephxSessionHandler: use connection_secret for encryption
msg,cephx: establish a unique connection_secret for every connection
msg/async: msgr2: use sha256_digest_t to print signature hex strings
types.h,rgw: merge sha*_digest_t definitions
msg/async: msgr2: close connection when no authorizer is given
msg/async: msgr2: formatting fixes
msg/async: msgr2: send client v2 address when only v1 address is defined
msg/async: msgr2: add payload length to banner
msg/async: msgr2: check protocol state after fast dispatch
msg/async: msgr2: reduce log level for sending messages event
msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used
msg/async: msgr2: store peer entity name in the protocol
msg/async: msgr2: apply sign/encrypt to messages data payload
msg/async: msgr2: encryption/decryption of frames
cephx: added encrypt/decrypt bufferlist method to session handler
msg/async: msgr2: refactored the frame structures
cephx: add sign bufferlist method
options: msgr2 enable/disable signing and encrytion options
msg/async: msgr2: cephx authentication
msg/async: msgr2: implement reconnect
msg/async: msgr2: fault handling
msg/async: msgr2: messange exchange phase
msg/async: msgr2: message flow handshake
msg/async: msgr2: authentication phase
msg/async: msgr2: exchange peer_type in banner phase
test/msgr: cloned test_msgr test for testing msgr2 protocol
msg/async: msgr2: banner exchange
msg/async: asyncconnection: update the source address info
msg/async: move base class Protocol its own source file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25583 from rhcs-dashboard/timepicker

mgr/dashboard: Adds custom timepicker for grafana iframes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #25745 from socketpair/precision

Formatters: improve precision of double numbers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: added 'env_build' to 'npm run e2e'

* 'e2e' & 'e2e:dev' need also to run 'env_build' to complete successfully.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #25725 from cfanz/wip-mgr-handle-monmap

mgr: update daemon_state when necessary

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26155 from tchaikov/wip-osd-cleanup

osd/PG: remove unused functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26156 from tchaikov/wip-use-latest-setuptools

install-deps.sh: use the latest setuptools

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26138 from Songweibin/wip-rbd-py-cleanup

pybind/rbd: fix a typo in metadata_get comments

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26105 from neha-ojha/wip-37886

mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26086 into master

* refs/pull/26086/head:
include/xlist: prevent item moves
include/xlist: privatize item members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: clear dispatch throttle on connection stop

This aligns us with the ProtocolV1 implementation.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #25616 from s0nea/wip-dashboard-osd-recv-speed-default-vals

mgr/dashboard: consider config option default values

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #24813 from wjwithagen/wjw-fix-alpine-compat

common: Fix Alpine compatability for TEMP_FAILURE_RETRY and ACCESSPERMS

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25916 from tchaikov/wip-debian-librados-dev

debian: librados-dev should replace librados2-dev

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26099 from jdurgin/wip-remove-ms-async-affinity

common/options: remove unused ms async affinity options

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26026 from tchaikov/wip-memstore-ceph-mutex

os/memstore: use ceph::mutex and friends

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: fix should_use_msgr2 behavior (including monc)

Be consistent about whether we should use v2 to connect or not, and fix
teh monclient check to use the same logic.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26159 from dzafman/wip-gitignore

git: Ignore tags anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: listen on random port in tests which start ceph-mon

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26150 from dillaman/wip-rados-lock-info

rados: return legacy address in 'lock info'

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

to avoid conflict with other `make check` runs in the same host

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #25599 into master

* refs/pull/25599/head:
mds: do not log internal journal error
mds: clean up log level 2 messages
mds: log session open stats

Reviewed-by: Zheng Yan <zyan@redhat.com>

git: Ignore tags anywhere

Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool cut down on large run

Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool: Add test for EC object dump to check hinfo section

Fixes: http://tracker.ceph.com/issues/38053
Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool: Fix EC code handling so it doesn't skip EC objects

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Remove unnecessary shell code that breaks this python test

Caused by: 8a694fc2f9e75864e064fe591feda9fab943c15e

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #26096 into master

* refs/pull/26096/head:
mds: remove cache drop asok command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #26087 into master

* refs/pull/26087/head:
mds: fix MDSGather inheritance
include: remove unnecessary inheritance

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26083 into master

* refs/pull/26083/head:
mds: trim cache after journal flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>