rpm: add dependency on python-kubernetes module to ceph-mgr-rook package

Signed-off-by: Jeff Layton <jlayton@redhat.com>

rpm,deb: add rbd_support module to ceph-mgr

Signed-off-by: Sage Weil <sage@redhat.com>

packaging: split ceph-mgr diskprediction and rook plugins into own packages

There are troublesome dependencies for both, so being able to deselect
them is useful

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #25542 from jan--f/c-v-extend-batch

Extend batch to accept explicit device lists

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26205 from cbodley/wip-38115

qa/mon: fix cluster support for monmap bootstrap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26204 from pritha-srivastava/wip-rgw-signature-mismatch

rgw: Fix for SignatureMismatchError in s3 commands.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #25403 from liyichao/rdigest

tools: Add clear-data-digest command to objectstore tool.

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #26158 from dzafman/wip-38053

Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/mon: fix cluster support for monmap bootstrap

-filter out mons from other clusters
-fix parsing of mon name from role

Fixes: http://tracker.ceph.com/issues/38115
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: Fix for SignatureMismatchError in s3 commands.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #25638 from ricardoasmarques/iscsi-management-w

mgr/dashboard: iSCSI management API

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26172 from rhcs-dashboard/fix-skipped-api-tests

mgr/dashboard: fix skipped backend API tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph-volume: don't crash on unusable data devices

Also add test case for a MixedStrategy with unusable data devices.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: add a wal device specific unit test and size validator

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: add osd_ids argument

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #26091 from tchaikov/wip-36737

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

Tested-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #26142 into master

* refs/pull/26142/head:
doc/dev/cephx: document current implementation of cephx

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #26097 from trociny/wip-rbd-mirror-perf-lock

rbd-mirror: guard access to image replayer perf counters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26168 from shyukri/qa-rbd-replace-bench-write

qa: rbd/workunits : Replace "rbd bench-write" with "rbd bench --io-type write"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: fix skipped backend API tests

* When the creation of the cluster is delegated to vstart_runner.py
(--create or --create-target-only) the amount of MGRs required
is calculated by the script so there is no more skipped tests
due to insufficient amount of MGRs.

* Additionally, this issue is not reproducible anymore:
Fixes: https://tracker.ceph.com/issues/37964
* Fixed typo: TEUTHOLOFY_PY_REQS

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #26162 from familyuu/crush

mgr/dashboard: CRUSH map viewer RFE

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #26182 from p-na/wip-pna-fix-pool-reload

mgr/dashboard: Fix reloading of pool listing

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #26143 from LenzGr/master-documentation

doc: Updated feature list and overview in dashboard.rst

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #25893 from sebastian-philipp/orchestrator-current-status

doc/orchestrator: Aligned Documentation with specification

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

mgr/dashboard: CRUSH map viewer RFE

Fixes: http://tracker.ceph.com/issues/37794
Signed-off-by: familyuu <guodan1@lenovo.com>

Merge pull request #26175 from neha-ojha/wip-37919

osd/PG.cc: account for missing set irrespective of last_complete

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #25737 from arthurhsliu/patch-1

doc: fix typo in erasure coding example

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/dev/cephx: document current implementation of cephx

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26157 from cbodley/wip-rgw-olh-repair

rgw: repair olh attributes that were broken by sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #26002 from pritha-srivastava/wip-sts-assumerole-webidentity

rgw: Initial commit for AssumeRoleWithWebIdentity.

Merge pull request #26110 from yaozongyou/fix-statistics-bug

mon: fix pg_sum_old not copied correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/orchestrator: Aligned Documentaiton with specification

Also: add current implementation status
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #26090 from tchaikov/wip-crimson-osdmap

crimson: persist/load osdmap to/from store

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Fix reloading of pool listing

Remove broken functionality that prevents pools from being reloaded after
deletion. The code also introduced a different problem; when clicking on a tab
shortly after the page has been loaded, the code will restore the tab to the
wrong one.

Introduced by 9e913bfd59c3a08c9e06bd8535bb7b92593ba549

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #26135 from Tinatinak/fix_permission_decorator

mgr/dashboard: Correct permission decorator

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26165 from rhcs-dashboard/ng-envbuild-e2e

mgr/dashboard: added 'env_build' to 'npm run e2e'

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

crimson/osd: fix boot process

* send my client addresses to mon
* always update osdmap with the latest one

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: load_pgs()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: add omap_get_values()

osd uses it for storing the pg_info.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/HitSet: mark copy ctor of HitSet::Params noexcept

to be returned using seastar::future<...> the value type should satisfy
std::is_nothrow_constructible<T>.

with this change, pg_pool_t will be nothrow_constructible. and hence
can be returned using seastar::future<pg_pool_t>. otherwise
std::is_nothrow_constructible<pg_pool_t>::value would be false.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add operator<< to OSDState

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: note missing coll/oid in exception

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: load/store osdmap from/to store

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26133 from dillaman/wip-37913

rbd: implement new 'rbd perf image iostat/iotop' commands

Reviewed-by: Mykola Golub <mgolub@suse.com>

crimson/osd: add "--mkfs" support to crimson-osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: move more features from MemStore to CyanStore

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Correct permission decorator

'CreatePermission' is required in order to restore an image from trash.

Signed-off-by: Tina Kallio <tina.kallio@gmail.com>

rgw: Initial commit for AssumeRoleWithWebIdentity.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

rbd: new 'perf image iostat/iotop' actions

This provides an 'iostat' and 'top'-like IO monitor for all
RBD images

Fixes: http://tracker.ceph.com/issues/37913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: add RGWRados::repair_olh() to reconstruct olh attrs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWRados::bi_get_olh() to read/decode olh entry

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: bi_get() takes bucket_info

and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add BucketShard::init overload for bucket_info,obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: simplify pool namespace positional argument handling

The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/rbd_support: support module for RBD features that require coordination

This initially adds mgr commands to collect RBD performance metrics
from the OSD dynamic perf collector:

  rbd perf image stats: collects IO rate statistics
  rbd perf image counters: collects raw IO counters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25112 from dzafman/wip-scrub-warning

scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26081 from cbodley/wip-38007

rgw: bucket full sync handles delete markers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #25137 from yehudasa/wip-archive-zone-3

rgw: archive zone

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

ceph-volume: add explicit mixed-type functional tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

rgw: bucket full sync handles delete markers

full sync uses CLS_RGW_OP_LINK_OLH_DM for delete markers instead of
trying to fetch them from the source zone

Fixes: http://tracker.ceph.com/issues/38007
Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add multisite test_suspended_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add multisite test_delete_marker_full_sync()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: use version_id from key directly

now that PutObj returns the x-amz-version-id, it's available through
Key.version_id without resorting to Bucket.list_versions()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: check_bucket_eq() supports delete markers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: check_bucket_eq() uses bucket.list_versions()

get_all_versions() is documented as a lower-level api that doesn't
handle paging, ands suggests list_versions() instead. also caches the
results to avoid listing each bucket twice

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25794 from cbodley/wip-rgw-sync-olh-attrs

rgw: fetch_remote_obj filters out olh attrs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

qa:rbd/workunits : Replace `rbd bench-write` with `rbd bench --io-type write`

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>

Merge PR #24546 into master

* refs/pull/24546/head:
msg/async/ProtocolV2: clear dispatch throttle on connection stop
msg/async: fix should_use_msgr2 behavior (including monc)
msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
ceph_test_msgr: do not connect_to on the client side
msg/async: do not connect from server
msg/async: do not use peer to addr detection; use getsockname()
msg/async/ProtocolV2: always send non-empty addrvec for self
msg/async: never fill out port in myaddr if we didn't bind
ceph_test_msgr: use v2 addrs for simplemessenger
msg/async: msgr2: don't force write event on every message received
msg/async/ProtocolV2: be forgiving in server identity check
msg/async/ProtocolV2: fault if we connect to the wrong peer
msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION
msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION
msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection
msg/async: msgr2: organizing log messages
msg/async: msgr2: fix connection fault when replacing
msg/async: msgr2: fix replacing race handling
msg/async: msgr2: fix connection race when existing connection is newer
msg/async: msgr2: assign recv_stamp in handle_message
msg/async: msgr2: fix peer_addrs discovery
msg/async: msgr2: keep authorizer bufferlist across reconnects
msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds
msg/async: msgr2: send keepalive on connection race winner
msg/async: msgr2: fix client address learning
msg/async: msgr2: fix keepalive_ack message
msg/async: msgr2: do not force updating rotating keys inline
msg/async: msgr2: fix mark_down vs accept race
msg/async: msgr2: unregister con from accept vs mark_down race
auth/cephx/CephxSessionHandler: use connection_secret for encryption
msg,cephx: establish a unique connection_secret for every connection
msg/async: msgr2: use sha256_digest_t to print signature hex strings
types.h,rgw: merge sha*_digest_t definitions
msg/async: msgr2: close connection when no authorizer is given
msg/async: msgr2: formatting fixes
msg/async: msgr2: send client v2 address when only v1 address is defined
msg/async: msgr2: add payload length to banner
msg/async: msgr2: check protocol state after fast dispatch
msg/async: msgr2: reduce log level for sending messages event
msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used
msg/async: msgr2: store peer entity name in the protocol
msg/async: msgr2: apply sign/encrypt to messages data payload
msg/async: msgr2: encryption/decryption of frames
cephx: added encrypt/decrypt bufferlist method to session handler
msg/async: msgr2: refactored the frame structures
cephx: add sign bufferlist method
options: msgr2 enable/disable signing and encrytion options
msg/async: msgr2: cephx authentication
msg/async: msgr2: implement reconnect
msg/async: msgr2: fault handling
msg/async: msgr2: messange exchange phase
msg/async: msgr2: message flow handshake
msg/async: msgr2: authentication phase
msg/async: msgr2: exchange peer_type in banner phase
test/msgr: cloned test_msgr test for testing msgr2 protocol
msg/async: msgr2: banner exchange
msg/async: asyncconnection: update the source address info
msg/async: move base class Protocol its own source file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25583 from rhcs-dashboard/timepicker

mgr/dashboard: Adds custom timepicker for grafana iframes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #25745 from socketpair/precision

Formatters: improve precision of double numbers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: added 'env_build' to 'npm run e2e'

* 'e2e' & 'e2e:dev' need also to run 'env_build' to complete successfully.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #25725 from cfanz/wip-mgr-handle-monmap

mgr: update daemon_state when necessary

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26155 from tchaikov/wip-osd-cleanup

osd/PG: remove unused functions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26156 from tchaikov/wip-use-latest-setuptools

install-deps.sh: use the latest setuptools

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26138 from Songweibin/wip-rbd-py-cleanup

pybind/rbd: fix a typo in metadata_get comments

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26105 from neha-ojha/wip-37886

mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26086 into master

* refs/pull/26086/head:
include/xlist: prevent item moves
include/xlist: privatize item members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: clear dispatch throttle on connection stop

This aligns us with the ProtocolV1 implementation.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #25616 from s0nea/wip-dashboard-osd-recv-speed-default-vals

mgr/dashboard: consider config option default values

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #24813 from wjwithagen/wjw-fix-alpine-compat

common: Fix Alpine compatability for TEMP_FAILURE_RETRY and ACCESSPERMS

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25916 from tchaikov/wip-debian-librados-dev

debian: librados-dev should replace librados2-dev

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26099 from jdurgin/wip-remove-ms-async-affinity

common/options: remove unused ms async affinity options

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26026 from tchaikov/wip-memstore-ceph-mutex

os/memstore: use ceph::mutex and friends

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: fix should_use_msgr2 behavior (including monc)

Be consistent about whether we should use v2 to connect or not, and fix
teh monclient check to use the same logic.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26159 from dzafman/wip-gitignore

git: Ignore tags anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: listen on random port in tests which start ceph-mon

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26150 from dillaman/wip-rados-lock-info

rados: return legacy address in 'lock info'

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

to avoid conflict with other `make check` runs in the same host

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #25599 into master

* refs/pull/25599/head:
mds: do not log internal journal error
mds: clean up log level 2 messages
mds: log session open stats

Reviewed-by: Zheng Yan <zyan@redhat.com>

git: Ignore tags anywhere

Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool cut down on large run

Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool: Add test for EC object dump to check hinfo section

Fixes: http://tracker.ceph.com/issues/38053
Signed-off-by: David Zafman <dzafman@redhat.com>

test: ceph-objectstore-tool: Fix EC code handling so it doesn't skip EC objects

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Remove unnecessary shell code that breaks this python test

Caused by: 8a694fc2f9e75864e064fe591feda9fab943c15e

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #26096 into master

* refs/pull/26096/head:
mds: remove cache drop asok command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #26087 into master

* refs/pull/26087/head:
mds: fix MDSGather inheritance
include: remove unnecessary inheritance

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26083 into master

* refs/pull/26083/head:
mds: trim cache after journal flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #26082 into master

* refs/pull/26082/head:
client: flush cap release on session flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

msg/async/AsyncMessenger: clear need_addr *after* we set our new addr

We check need_addr at the top without a lock held, so we need to be sure
we finished our work before we clear it, or else when there are two racing
threads the first will get the clock and clear the value and the second
will do nothing and see the unlearned value before the first finishes.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr

Switch it to be v2.  Reject the case where the client sends and addrvec, though;
that should only happen for clients that did_bind, and they should only connect to
v2 if they have a v2 bound addr.

Signed-off-by: Sage Weil <sage@redhat.com>