osd/OSD: Set the number of cache shards independently.  Default to 32

Signed-off-by: Mark Nelson <mnelson@redhat.com>

os/bluestore: Keep separate onode cache pinned list.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

mgr/dashboard: REST API: OpenAPI docs require internet connect… (#30477)

mgr/dashboard: REST API: OpenAPI docs require internet connection

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30557 from smithfarm/wip-42023

install-deps.sh: add EPEL repo for non-x86_64 archs as well

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #30590 from jtlayton/wip-42032

ceph.spec.in: fix Cython package dependency for Fedora

Merge pull request #29460 from rhcs-dashboard/bucket-versioning

mgr/dashboard: enable/disable versioning on RGW bucket

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30208 from ricardoasmarques/support-iscsi-controls-types

mgr/dashboard: Controls UI inputs based on "type"

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #28834 into master

* refs/pull/28834/head:
libcephfs: Add test for lazyio via libcephfs
libcephfs: Add lazyio_propogate and lazyio_synchronize methods to libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #29818 into master

* refs/pull/29818/head:
client/MetaRequest: Add age to MetaRequest dump
osdc/Objecter: Add age to the ops
common/ceph_time: Use fixed floating-point notation for mono_clock

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29902 into master

* refs/pull/29902/head:
mds: Revert "properly setup client_need_snapflush for snap inode"
mds: cleanup dirty snap caps tracking

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29921 into master

* refs/pull/29921/head:
mds: split the dir if the op makes it oversized

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #30202 into master

* refs/pull/30202/head:
mds: Explicitly call slave_updates with 0 size
mds: Move log_segment_seq_t into class LogSegment
mds: Reorganize class members in LogSegment header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30403 into master

* refs/pull/30403/head:
mds: mds returns -5 error when the deleted file does not exist

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #30462 into master

* refs/pull/30462/head:
qa: have kclient tests use new mount.ceph functionality

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/dashboard: Support iSCSI target-level CHAP auth (#30011)

mgr/dashboard: Support iSCSI target-level CHAP auth

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30588 from ricardoasmarques/fix-iscsi-test-put

mgr/dashboard: Wait for iSCSI target put and delete

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: fix Cython package dependency for Fedora

Fedora distros do not have python3?-Cython packages, but they do have
python3-Cython ones. Fix the BuildRequires so that we only use the
python3_version_nodots based version string for RHEL.

Fixes: https://tracker.ceph.com/issues/42032
Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/dashboard: Wait for iSCSI target put and delete

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #30571 from smithfarm/wip-bci-force

src/script/backport-create-issue: implement --force option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Controls UI inputs based on "type"

iSCSI controls UI inputs should be rendered based
on the control "type" returned by `ceph-iscsi`.

Fixes: https://tracker.ceph.com/issues/41682
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: test_mgr_module QA test has not been adapted to… (#30513)

mgr/dashboard: Adapt test_mgr_module QA test to latest controller changes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: wait for iscsi/target create/put (#30582)

mgr/dashboard: wait for iscsi/target create/put

Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: enable/disable versioning on RGW bucket

Fixes: https://tracker.ceph.com/issues/40920
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #30578 from tchaikov/wip-seastar-unix-domain-socket

seastar: pickup unix domain socket support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

mgr/dashboard: wait for iscsi/target create/put

IscsiTarget.create() returns a Task, so we need to wait for its
completion before moving on.

the same applies to IscsiTarget.put()

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: pickup unix domain socket support

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #30523 into master

* refs/pull/30523/head:
mon/MonClient: skip CEPHX_V2 challenge if client doesn't support it

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Support iSCSI target-level CHAP auth

Requires `ceph-iscsi` config v11

Fixes: https://tracker.ceph.com/issues/41576
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Display the "destroyed" state in OSD list (#30514)

mgr/dashboard: Display the "destroyed" state in OSD list

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

install-deps.sh: add EPEL repo for non-x86_64 archs as well

Fixes: https://tracker.ceph.com/issues/42023
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

src/script/backport-create-issue: implement --force option

If --force option is given along with an explicit list of issue numbers,
backport issues will be created regardless of issue status.

In all other usage scenarios, script behavior remains the same (backport
issues are created only if issue status is "Pending Backport").

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30420 from runsisi/wip-remove-requests

rpm,deb: python-requests is not needed for ceph-common

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30422 from rishabh-d-dave/cv-fix-pytest-warning

ceph-volume: fix warnings raised by pytest

Merge pull request #30036 from tchaikov/wip-python3-pkgversion

ceph.spec.in: s/pkgversion/version_nodots/

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

ceph.spec.in: s/pkgversion/version_nodots/

`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Gracefully handle client/target info not found (#30349)

mgr/dashboard: Gracefully handle client/target info not found

Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Using wrong identifiers in RGW user/bucket data… (#30492)

mgr/dashboard: Using wrong identifiers in RGW user/bucket datatables

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #30537 into master

* refs/pull/30537/head:
test/pybind: fix local variables for emacs
crush: add editor property variables

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Gracefully handle client/target info not found

Fixes: https://tracker.ceph.com/issues/41779
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #29824 into master

* refs/pull/29824/head:
qa: whitelist new FS_INLINE_DATA_DEPRECATED health warning
mds: add a HEALTH_WARN message when inline_data is enabled
mds: log a warning message when mds is started on an fs with inline_data
mon: deprecate CephFS inline_data support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge PR #29526 into master

* refs/pull/29526/head:
client: _readdir_cache_cb() may use the readdir_cache already clear

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #30426 from votdev/harden_mgr_modules_list

mgr/dashboard: Prevent KeyError when requesting always_on_modules

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge PR #30164 into master

* refs/pull/30164/head:
mds: Remove unnecessary headers in Locker
mds: Reorganize class members in Locker header

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30312 into master

* refs/pull/30312/head:
client:EINVAL may be returned when offset is 0 ,Loff_t pos = f->pos should be the best.

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30493 from p-se/wip-pse-osd-improvements

mgr/dashboard: OSD improvements

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: enable run-frontend-unittest.sh to run from any… (#30496)

mgr/dashboard: enable run-frontend-unittest.sh to run from any directory

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix error editing iSCSI disk controls (#30100)

mgr/dashboard: Fix error editing iSCSI disk controls

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #30535 from xiexingguo/wip-test_ceph_argparse

test/pybind/test_ceph_argparse.py: pg_num of pool creation now optional

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/pybind: fix local variables for emacs

* pass CEPH_BIN env variable if necessary
* do not 'make' unless necessary
* use `cmake --build` as developer might be using some different cmake
  generator for building ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crush: add editor property variables

so the local settings won't interfere when working on ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/pybind/test_ceph_argparse.py: pg_num of pool creation now optional

introduced by f88d95bef6948410603bd877aa4714a2afefd5a5

note that we remove the bad pg_num = -1 test too, which will now
be treated as the the crush rule name (invalid but allowed) since
pg_num is optional

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #30348 into master

* refs/pull/30348/head:
doc: Add a new document on Dynamic Metadata Management in CephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30512 from xiexingguo/wip-vstart-inc-osd

vstart: add --inc-osd option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30490 from ceph/guits-41392

ceph-volume: do not fail when trying to remove crypt mapper

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

mon/MonClient: skip CEPHX_V2 challenge if client doesn't support it

If the client doesn't support the CEPHX_V2 challenge, and we don't require
it, skip it.  This allows the client to authenticate without getting an
error like

  cephx: verify_reply couldn't decrypt with error: error decoding block for decryption

Note that we don't have this problem in the monitor exchange in
Monitor::handle_auth_request() because that verify_authorizer() caller is
only used for msgrv2, and all such clients support CEPHX_V2.  Instead,
those client authenticate via the MAuth messages, a path that does not use
authorizers at all.

Fixes: https://tracker.ceph.com/issues/40716
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Display the "destroyed" state in OSD list

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30507 from tchaikov/wip-mgr-features-in-mgrmap

qa/tasks/mgr/dashboard/test_health: update schema

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

doc: Add a new document on Dynamic Metadata Management in CephFS

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

Merge PR #30475 into master

* refs/pull/30475/head:
qa/standalone/ceph-helpers: default pg autoscale mode off for standalone
os/bluestore: fix objectstore_blackhole read-after-write
test,misc: do not specify pg_num per pool
mgr/volumes: do not specify pg_num
pybind/ceph_volume_client: do not specify pg_num for new pools
doc: remove all pg_num arguments to 'osd pool create'
mon: do not require pg_num to 'osd pool create'
common: default pg_autoscale_mode=on for new pools

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #30503 into master

* refs/pull/30503/head:
osd/PeeringState: recover_got - add special handler for empty log

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #30506 into master

* refs/pull/30506/head:
qa/standalone/osd/divergent-priors: add reproducer for bug 41816

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/dashboard: Fix spacing between cards (#30461)

mgr/dashboard: Fix spacing between cards

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: test_mgr_module QA test has not been adapted to latest controller changes

Fixes: https://tracker.ceph.com/issues/41993
Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Improve and simplify OsdListComponent

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: fix error when linking a bucket to a tenanted u… (#30123)

mgr/dashboard: fix error when linking a bucket to a tenanted user

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: OSD list component improvements (#30488)

mgr/dashboard: OSD list component improvements

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: NFS list should display the "Pseudo Path" (#30395)

mgr/dashboard: NFS list should display the "Pseudo Path"

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

ceph-volume: do not fail when trying to remove crypt mapper

In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.

Closes: https://tracker.ceph.com/issues/41392
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #30411 from dillaman/wip-41653

rbd-mirror: simplify peer bootstrapping

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #30428 into master

* refs/pull/30428/head:
doc/dev/developer_guide: fix heading level

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30435 into master

* refs/pull/30435/head:
doc: improve developer guide doc

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30497 into master

* refs/pull/30497/head:
ceph-volume: systemd fix typo in log message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rpm,deb: python-requests is not needed for ceph-common

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>

vstart: add --inc-osd option

I am currently developing some scale-up[down] related feature.
It would be nice if I can simply develop and test it on the
vcluster at the early stage.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/standalone/ceph-helpers: default pg autoscale mode off for standalone

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix objectstore_blackhole read-after-write

Instead of tossing out transactions from queue_transactions (which prevents
us from reading back the results), instead (1) stop doing any write IO, and
(2) stop trimming the cache.

Fixes: https://tracker.ceph.com/issues/40684
Signed-off-by: Sage Weil <sage@redhat.com>

test,misc: do not specify pg_num per pool

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/volumes: do not specify pg_num

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/ceph_volume_client: do not specify pg_num for new pools

Signed-off-by: Sage Weil <sage@redhat.com>

doc: remove all pg_num arguments to 'osd pool create'

Also, update the dicussion about pg_num and pool creation, with a reference
to the autoscaler.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: do not require pg_num to 'osd pool create'

Signed-off-by: Sage Weil <sage@redhat.com>

common: default pg_autoscale_mode=on for new pools

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30457 from dillaman/wip-41908

osd/PrimaryLogPG: do not append outdata to TMAPUP ops

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/mgr/dashboard/test_health: update schema

see 351a3b9d3867ab8dd6d0502fbc46c0016553c6d5

Fixes: https://tracker.ceph.com/issues/41947
Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MgrMap: dump mgr_features as an object property

it's a property of "standby" object, not an element of
"available_modules".

it's a follow-up fix of 351a3b9d

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #30427 into master

* refs/pull/30427/head:
doc/cephfs: correct a description mistake about mds states

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30432 from yanghonggang/wip-yhg-balancer-eval-maxmin

pybind/mgr/balancer/module.py: add max/min info in stats_by_root

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30479 from liewegas/wip-fix-osd-invalid-stats

osd/osd_types: fix {omap,hitset_bytes}_stats_invalid handling on split/merge

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #29800 from Devp00l/wip-41372

mgr/dashboard: CephFS tab component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30412 from tchaikov/wip-crimson-osd-make-message

osd,crimson: use make_message for creating message

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30408 from p-se/wip-pna-smartctl-report

osd: add common smartctl output to JSON output

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30414 from neha-ojha/wip-fix-missing

osd/PeeringState.h: get_num_missing() should report num_missing()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30448 from tchaikov/wip-40871

mgr: set hostname in DeviceState::set_metadata()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30449 from tchaikov/wip-37884

mon,mgr: pass MessageRef to monc.send_mon_message()  …

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30413 from vumrao/wip-vumrao-41875

tools/rados: call pool_lookup() after rados is connected.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30409 from uweigand/cleanup-endian

src/: Clean up endian handling

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30383 from vumrao/wip-vumrao-tools-rados-add-pgid-in-help

tools/rados: add --pgid in help

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30211 from majianpeng/msg-clear-payload

msg/async: clear_payload when requeue_sent.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #30445 from tchaikov/wip-41320

mgr/dashboard: do not log tokens

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #29947 from iotcg/nosrq

support RDMA NIC without SRQ in msg/async/rdma

Reviewed-by: Roman Penyaev <rpenyaev@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>