mgr/dashboard: fix RGW subuser auto-generate key.

Fixes: https://tracker.ceph.com/issues/43263
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #32176 from idryomov/wip-doc-fancy-striping

doc: clarify support for rbd fancy striping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #32163 into master

* refs/pull/32163/head:
qa: fix output check to not be sensitive to debugging

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #32083 from smithfarm/wip-42612

rpm: add rpm-build to SUSE-specific make check deps

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32160 into master

* refs/pull/32160/head:
doc/releases/nautilus,PendingReleaseNotes: consolidate telemetry note

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32161 from mgfritch/wip-cd-tox-warn

ceph-daemon: `imp` module  DeprecationWarning

mgr/dashboard: Disable cache for static files (#31018)

mgr/dashboard: Disable cache for static files

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc: clarify support for rbd fancy striping

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #32025 from tchaikov/wip-pick-address-silence-warning

common/pick_address.cc: silence GCC warning

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #32091 into master

* refs/pull/32091/head:
common/config: update values when they are removed via mon

Reviewed-by: Neha Ojha <nojha@redhat.com>

common/pick_address.cc: silence GCC warning

* use `string_view` instead of string
* use `auto` instead of `int` to silence the warning
* use `enum class` instead of plain `enum`
* do not specify the value of enum, as
  we don't care about it

this change silences following warning

```
../src/common/pick_address.cc: In function ‘int get_iface_numa_node(const string&, int*)’:
../src/common/pick_address.cc:517:11: warning: comparison of integer expressions of different signedness: ‘int’ and ‘const size_type’ {aka ‘const long unsigned int’} [-Wsign-com\
pare]
  517 |   if (pos != string::npos ) {
      |       ~~~~^~~~~~~~~~~~~~~
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31839 from alimaredia/wip-s3-test-branch-refactor

qa: update s3-test download code for s3-test tasks

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa: fix output check to not be sensitive to debugging

The traceback would screw up the equality checks.

Also, use unittest asserts so we can easily see the assert values.

Fixes: https://tracker.ceph.com/issues/43247
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31772 into master

* refs/pull/31772/head:
qa/standalone/test_ceph_daemon.sh: clone corpus explicitly
ceph-daemon: re-enable the OSD standalone test
ceph-daemon: move standalone test tgz to corpus
ceph-daemon: add ceph-daemon-adoption-corpus.git submodule

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30548 from trociny/wip-41557

rbd-mirror: snapshot mirror mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #31122 into master

* refs/pull/31122/head:
mds: Reorganize class members in MDSTable header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31253 into master

* refs/pull/31253/head:
mds: Reorganize class members in Migrator header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31596 into master

* refs/pull/31596/head:
mds: Reorganize class members in PurgeQueue header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-daemon: `imp` module  DeprecationWarning

Signed-off-by: Michael Fritch <mfritch@suse.com>

doc/releases/nautilus,PendingReleaseNotes: consolidate telemetry note

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32148 into master

* refs/pull/32148/head:
doc/releases/nautilus: fix release notes (crash->device)

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #32041 from dzafman/wip-balancer3-pending

release note: Add pending release notes for already merged code

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/standalone/test_ceph_daemon.sh: clone corpus explicitly

When this is run by teuthology we don't have a full ceph source tree
checkout with submodules.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: re-enable the OSD standalone test

Signed-off-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: move standalone test tgz to corpus

Fixes: https://tracker.ceph.com/issues/42876
Signed-off-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: add ceph-daemon-adoption-corpus.git submodule

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #32127 into master

* refs/pull/32127/head:
mgr/rook: Added Mypy static type checking

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32138 into master

* refs/pull/32138/head:
ceph-daemon: combine SUDO and ARGS into a single var

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32137 into master

* refs/pull/32137/head:
ceph-daemon: don't deref symlinks during chown

Reviewed-by: Sage Weil <sage@redhat.com>

qa: update s3-test download code for s3-test tasks

- Ensure the download code for all tasks running
s3-tests is consistent.
- Simplify download code to only use the config
variable 'force-branch' for the branch being
cloned.
- make ceph-master the force-branch for all
suites using s3-tests.

Fixes: https://tracker.ceph.com/issues/43077
Signed-off-by: Ali Maredia <amaredia@redhat.com>

rpm: add rpm-build to SUSE-specific make check deps

The binary /usr/bin/rpmspec was recently moved to rpm-build, breaking
install-deps.sh on openSUSE Tumbleweed. The package is not strictly
needed for SLE-15-SP* and openSUSE Leap 15.*, but it doesn't hurt to
have it, and will future-proof these distros from this regression.

Putting the dependency in the spec file does not address the issue,
because /usr/bin/rpmspec must be available before install-deps.sh runs it to
determine the dependencies, but it's nice to have it explicitly listed there,
since it *is* a dependency of "make check" on SUSE distros.

SUSE versions < 15 are of no interest in master/octopus+.

Fixes: https://tracker.ceph.com/issues/42612
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #32145 into master

* refs/pull/32145/head:
doc: show how to count jobs before triggering them

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32140 from tchaikov/wip-cmake-cleanup

cmake: silence messages when cppcheck/IWYU is not found

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #31954 from dzafman/wip-42346

osd: Use physical ratio for nearfull (doesn't include backfill resserve)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: force promote for snapshot mirroring

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: introduce LIST_WATCHERS_MIRROR_INSTANCES_ONLY flag

Also, don't skip listing mirror watchers if the image has journaling
disabled -- it is not correct for snapshot mirror mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: refactor CreateNonPrimaryRequest and CreatePrimaryRequest

to extract re-usable functions into utils

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: store/remove image state when creating/removing mirror snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: requests for mirror snapshot image state

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: async methods for snapshot_get_limit

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd: list primary mirror snapshots in mirror image status

TODO: provide the same in verbose mirror pool status.

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd: add 'mirror image snapshot' command

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: API method to get mirror image mode

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: add mirror image snapshot tests

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: add mirror image snapshot API

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: when removing mirror peer unlink it from all snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: snapshot mirror mode

Enabling mirroring for an image that does not support journaling
assumes snapshot based mirroring, which is supported only when the
pool is in the "image" mirror mode.

Also for the pool in the "image" mirror mode disabling/enabling
journaling feature for a mirroring image will switch
snapshot/journal mirror mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: state machine for snapshot based mirror promote/demote

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: state machine for creating mirroring snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: state machine for unlinking peer from mirror snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: method to set mirror snapshot copy progress

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: method to unlink peer from mirror snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: async methods for mirror_peer_list

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: cleanup: remove unused function

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: add mode (journal/snapshot) to mirror image

Signed-off-by: Mykola Golub <mgolub@suse.com>

ceph-daemon: don't deref symlinks during chown

- consolidate move/copy loop logic
- chown the actual file and not the symlink

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #32132 into master

* refs/pull/32132/head:
osd/OSDMap: rename old calc_pg_role -> calc_pg_role_broken
osd/OSDMap: remove dead osd_is_valid_op_target
osd/OSDMap: fix get_pg_acting_role()
osd: use spg_t for pending_creates_from_osd
osd/OSDMap: drop unused get_pg_acting_role()
osd/OSDMap: fix+simplify is_up_acting_osd_shard
osd: use new and improved calc_pg_role()
osd/OSDMap: new calc_pg_role() that takes a pg_shard_t
osd/OSDMap: calc_pg_rank -> calc_pg_role
osd/PeeringState: debug lines for upacting_features, proc_lease
osd/PeeringState: use pg_vector_string for operator<<

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/releases/nautilus: fix release notes (crash->device)

Fix pending note for octopus too

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31827 into master

* refs/pull/31827/head:
mgr/ssh: add 'upgrade check' command
ceph-daemon: add 'pull' command
mgr/orchestrator_cli: document orchestrator option
mgr/ssh: move entity name munging into a helper

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #31970 into master

* refs/pull/31970/head:
doc/releases: add Nautilus 14.2.5 to release timeline
doc/releases: add release notes for nautilus v14.2.5

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32141 from tchaikov/wip-doc-highlight-console

doc: use the console lexer for rendering command line sessions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31965 from rosinL/wip-osd-support-bond-network

osd/osd: Refactor get_iface_numa_node

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27392 from rouming/bluestore-iouring

os/bluestore: introduce new io_uring IO engine

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31979 from jan--f/c-v-patch-volumegroups

ceph-volume/test: patch VolumeGroups

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #32122 from s0nea/wip-dashboard-fix-tests

qa/tasks/mgr/dashboard/test_health: update schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

doc: show how to count jobs before triggering them

Suggesting to use teuthology-suite command switch --dry-run for that.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

os/bluestore: introduce new io_uring IO engine

This implements low-level IO engine, which utilizes brand-new
io_uring IO interface: https://lwn.net/Articles/776428/

By default libaio is used.  If bluestore_ioring=true is set but kernel
does not support io_uring or architecture is not x86-64, libaio will be
used instead.

In current patch liburing library is used in order not to open code
everything.

In order to compile with liburing WITH_LIBURING=ON should be specified.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>

os/bluestore: make abstract interface for any sorts of AIO queues

In the next patch new io_uring API will be used instead of libaio.
So this prepares the abstract interface.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>

Merge pull request #32007 from tchaikov/wip-osd-cleanup

osd: use unique_ptr for managing life cycles

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #31931 from majianpeng/bluestore-bugfix

os/bluestore: fix wakeup bug

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #32052 from mgfritch/wip-cd-standalone-tempfiles

ceph-daemon: clean-up tempfiles on EXIT

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #32100 from yehudasa/wip-43184

qa: radosgw-admin: remove dependency on bunch package

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #31961 from joscollin/wip-B43038-subvolume-group-create-fail

mgr/volumes: check for string values in uid/gid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: use console lexer for rendering command line sessions

by default the pygements uses python lexer for rendering literal blocks,
and `#` is considered as the marker for inline comments. in this change,
the "console" lexer is specified explicitly. see
https://pygments.org/docs/lexers/#lexers-for-various-shells for more details,
in which "BashSessionLexer" is for this very use case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #32124 into master

* refs/pull/32124/head:
doc/cephfs/disaster-recovery-experts: Add link for scrub and note for scrub_path

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cmake: silence messages when cppcheck/IWYU is not found

* remove messages spit when these tools are not found
  they are not necessary for building Ceph. and the messages in the cmake
  output are distracting.
* fix a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27894 into master

* refs/pull/27894/head:
cephfs-shell: Add default list to df
cephfs-shell: Add tests for df command
cephfs-shell: Fix 'df' command errors

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32131 into master

* refs/pull/32131/head:
codeowners: Add ceph2.py to @ceph/orchestrators

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32038 into master

* refs/pull/32038/head:
doc: reorganize CephFS landing page and ToC

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #32098 into master

* refs/pull/32098/head:
ceph-daemon: py2: tolerate whitespace before config key name

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #32037 from smithfarm/wip-43152-workaround

build/ops: make-dist: default to no dashboard frontend build parallelism

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/releases: add Nautilus 14.2.5 to release timeline

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/releases: add release notes for nautilus v14.2.5

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32088 from idryomov/wip-krbd-exclusive-lock-fixups

qa: krbd_exclusive_option.sh: update for recent kernel changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #32046 into master

* refs/pull/32046/head:
mgr/DaemonServer: fix 'osd ok-to-stop' for EC pools

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #31489 into master

* refs/pull/31489/head:
pybind/rados: add WriteOp::writesame() and test WriteOp::writesame()
pybind/rados: add Ioctx::aio_writesame() and test Ioctx::aio_writesame()
pybind/rados: add Ioctx::writesame() and test Ioctx::writesame()

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge PR #29214 into master

* refs/pull/29214/head:
mgr: templatize/generalize metrics collection interface

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32102 into master

* refs/pull/32102/head:
mgr/diskprediction_local: import scipy early to fix self-test deadlock

Reviewed-by: Kefu Chai <kchai@redhat.com>

release note: Add pending release notes for already merged code

Follow on to https://github.com/ceph/ceph/pull/31774

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #31794 from dillaman/wip-42889

librbd: disable zero-copy writes by default

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #32105 from junxfl/test1

librbd: fix rbd_open_by_id, rbd_open_by_id_read_only

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32109 from yehudasa/wip-42868

rgw: list buckets: don't return buckets if limit=0

osd/OSDMap: rename old calc_pg_role -> calc_pg_role_broken

Also primary_changed -> primary_changed_broken.

Logged this issue with https://tracker.ceph.com/issues/43213

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: combine SUDO and ARGS into a single var

- reduce the amount of typing/noise for each CEPH_DAEMON invocation
- ensure the `--image` param is passed to each test invocation
- allow passing additional args to ceph-daemon via CEPH_DAEMON_ARGS

Signed-off-by: Michael Fritch <mfritch@suse.com>

osd/OSDMap: remove dead osd_is_valid_op_target

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: fix get_pg_acting_role()

This needs to take an spgid in order to know what role the osd is
serving.

Signed-off-by: Sage Weil <sage@redhat.com>

codeowners: Add ceph2.py to @ceph/orchestrators

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #32053 from votdev/left_align_badges

mgr/dashboard: Left align badge datatable columns

Reviewed-by: Laura Paduano <lpaduano@suse.com>

osd: use spg_t for pending_creates_from_osd

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: drop unused get_pg_acting_role()

No callers.

Signed-off-by: Sage Weil <sage@redhat.com>