Merge pull request #31436 from dang/wip-dang-zipper-3

rgw: Project Zipper - Bucket

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #30754 into master

* refs/pull/30754/head:
doc/cephfs: merge fstab doc with respective mount docs
doc: add systemd unit part for FUSE mounts in fstab doc
doc: update and improve "mount using kernel driver" doc
doc: update and improve "mount using FUSE" doc

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31631 into master

* refs/pull/31631/head:
ceph-daemon: Add more type hints

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #31561 from sebastian-philipp/orchestrator-remove-cli-tox

mgr/orchestrator_cli: remove tox and move test to parent dir

Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/orchestrator: remove tox and move test to parent dir

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

ceph-daemon: Add more type hints

Commit 7ce26be7a3448 started to add type hints. This commit continues
that and adds the missing type hints.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

doc/cephfs: merge fstab doc with respective mount docs

Fixes: https://tracker.ceph.com/issues/42601
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #31614 into master

* refs/pull/31614/head:
ceph-daemon: use `-e` instead of `--env`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31613 into master

* refs/pull/31613/head:
mgr/ssh: fix redeploy

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #31609 into master

* refs/pull/31609/head:
ceph-daemon: Add basic mypy support

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #31601 from tchaikov/wip/install-newer-binutils-on-xenial

install-deps.sh: install binutils 2.28 for xenial

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge PR #31602 into master

* refs/pull/31602/head:
mgr/ssh: annotate object representation

Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph-daemon: use `-e` instead of `--env`

Older versions of podman don't allow intermixing `-e` and `--env`

Signed-off-by: Michael Fritch <mfritch@suse.com>

install-deps.sh: remove the cleanup for kitware repo

it was used for removing the kitware repo, and in
7265b55d094a639be50a567d3be92fba94c04786, we switched to a rebuilt
version hosted in chacra, and it was more than two months ago,
presumably, none builder has kitware repo anymore. so, let's remove
the cleanup code.

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: install binutils 2.28 for xenial

ld 2.26 segment faults when linking librados.so in
https://github.com/ceph/ceph/pull/16715 .

while ld 2.28 always survives.
binutils 2.28 is rebuilt and uploaded to chacra
binutils/master/7fa393306ed8b93019d225548474c0540b8928f7/ubuntu/xenial,

so let's use it instead.

the source deb package comes from
http://ppa.launchpad.net/jonathonf/binutils/ubuntu

i tried to use the binutils 2.30 source package from bionic,
but it has build dependency of dpkg-dev (>= 1.19.0.5) which cannot be
fulfilled on xenial. so without more changes, we cannot get
binutils 2.30 built on xenial using this source package.

Fixes: https://tracker.ceph.com/issues/42596
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31592 from jan--f/c-v-fix-mimic-functional-tests

ceph-volume: assume msgrV1 for all branches containing mimic

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #31603 into master

* refs/pull/31603/head:
ceph-daemon: fix prompt

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

ceph-daemon: Add basic mypy support

Add mypy support and some type hints to be able to do static code
analysis.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

mgr/ssh: fix redeploy

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Add E2E tests for tasks and notifications sideb… (#31128)

mgr/dashboard: Add E2E tests for tasks and notifications sidebar

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

ceph-daemon: fix prompt

[ceph: root@gnit /]#
not
[ceph: root@gnit /]$

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: annotate object representation

* InventoryNode
* ServiceDescription

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge PR #31583 into master

* refs/pull/31583/head:
ceph-daemon: -v|--verbose, not -d|--debug

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #31585 into master

* refs/pull/31585/head:
ceph-daemon: add `--legacy-dir` arg to `ls` command

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31533 from tspmelo/wip-bc-redmine-key

backport-create-issue: read redmine key from file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #31544 from ifed01/wip-ifed-gettid2

test/fio: bump to fio-3.15

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-volume: assume msgrV1 for all branches containing mimic

With nautilus and newer OSDs listen on v1 ports and v2 ports. Assume
that if mimic (or luminous) occur in the branch name, the OSDs are
running msgrv1 only.

Fixes: https://tracker.ceph.com/issues/42791
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #31426 from cyx1231st/rfc-seastar-fix-build-dpdk

crimson: fix build seastar with dpdk

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30662 from sebastian-philipp/orchestrator-c-v-inventory

python-common, mgr/orchestrator, mgr/dashboard: Use common "Devices"

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

crimson: build seastar dpdk from src/seastar/dpdk

src/spdk/dpdk and src/seastar/dpdk are both at their private branches
with project-specific modifications, so select proper dpdk source
directory according to flags WITH_SPDK and Seastar_DPDK.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson: link required dpdk components for seastar

see https://github.com/scylladb/seastar/blob/
809b3ef875e7edb7c56643812dc8347ff9a82cee/cmake/Finddpdk.cmake#L60-L85

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

doc: add systemd unit part for FUSE mounts in fstab doc

To make FUSE-mounted CephFS persist across reboots, user also needs to
start and enable the systemd units. Add that part to the document for
fstab, instead of mentioning it in "Mount CephFS using FUSE" doc. Also,
wrap few lines and rename mountpoint to /mnt/mycephfs in examples to
keep them same across docs.

Fixes: https://tracker.ceph.com/issues/42298
Signed-off-by: Rishabh Dave <ridave@redhat.com>

doc: update and improve "mount using kernel driver" doc

Move the examples of mount command for Ceph cluster with CephX enabled
to the top of page, since it is enabled by default, improve explanation
around Ceph with multiple FSs, get rid of hash symbols before every
command (without them it's clear that the text is command and with them
the reader cannot use the commands directly), link fstab page, add how
mount in general looks, add prerequisites required for kernel mounts and
expand explanation wherever possible.

Fixes: https://tracker.ceph.com/issues/42220
Signed-off-by: Rishabh Dave <ridave@redhat.com>

doc: update and improve "mount using FUSE" doc

Recommend keyring permission to be 600 instead of 644, show examples
for `-k`, `-r`, `-m` and `--client_mds_namespace` options, move
references to the bottom of the page,  show how to unmount FUSE-mounted
CephFS, copy the tip about unmounting from "mount using kernel" page to
"mount using FUSE" page, correct commands for automating FUSE mounts,
add sub-headings to the document and add how ceph-fuse command looks in
general.

Fixes: https://tracker.ceph.com/issues/42205
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #31571 from rhcs-dashboard/42775-fix-typo

common: fix typo in rgw_user_max_buckets option long description.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31510 from yuvalif/wip-yuval-remove-flaky-amqp-test

rgw/amqp: remove flaky amqp test

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-daemon: add `--legacy-dir` arg to `ls` command

Signed-off-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: -v|--verbose, not -d|--debug

-v is more natural IMO

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31575 into master

* refs/pull/31575/head:
ceph-daemon: 'logs' command

Reviewed-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: 'logs' command

This just invokes podman logs with the appropriate argument(s).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31577 into master

* refs/pull/31577/head:
pybind/mgr/CMakeLists: exclude tox.ini, requirements.txt from install

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31542 from neha-ojha/wip-42756

osd/OSDMap.cc: don't output over/underfull messages to lderr

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>

pybind/mgr/CMakeLists: exclude tox.ini, requirements.txt from install

Signed-off-by: Sage Weil <sage@redhat.com>

backport-create-issue: read redmine key from file

Now you don't need to provide redmine key or user/password as a flag, as the
script will read it from "~/.redmine_key".

This makes it more consistent with other scripts.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge PR #31157 into master

* refs/pull/31157/head:
doc: update inf/infinite option for subvolume resize
qa/tasks: fs subvolume resize inf command
mgr/volumes: fs subvolume resize inf/infinite command

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31503 into master

* refs/pull/31503/head:
Revert "os/bluestore: add kv_drain_preceding_waiters indicate drain_preceding."

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #31537 into master

* refs/pull/31537/head:
mgr/orchestrator: move split_host into shared helpers
mgr/ssh: allow names to be specified for stateless services
mgr/ssh: optionally specify names for mon daemons
mgr/ssh: optionally specify names for mgr daemons

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #31280 from zy751713126/snap_get_name_and_id

librbd: add snap_get_name and snap_get_id method API

Merge pull request #31491 from zy751713126/rbd_bench

rbd/bench: outputs bytes/s format dynamically

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31497 from trociny/wip-mock-warnings

test/librbd: fix mock warnings in TestMockIoImageRequest

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #31487 into master

* refs/pull/31487/head:
ceph-daemon: add `--base-dir` arg to `adopt` command

Reviewed-by: Sage Weil <sage@redhat.com>

common: fix typo in rgw_user_max_buckets option long description.

https://tracker.ceph.com/issues/42775
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/orchestrator: move split_host into shared helpers

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: allow names to be specified for stateless services

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: optionally specify names for mon daemons

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: optionally specify names for mgr daemons

Use '=' as a separator.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Manage PG autoscaling (#31417)

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #31559 into master

* refs/pull/31559/head:
mgr: Python cleanup and type check

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31539 into master

* refs/pull/31539/head:
mgr/orchestrator_cli: sort by service name
mgr/orchestrator_cli: collapse TYPE and ID columns

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #31394 into master

* refs/pull/31394/head:
mgr/crash: Calculate and add stack_sig to metadata

Reviewed-by: Sage Weil <sage@redhat.com>

mgr: Python cleanup and type check

- Add/remove missing/obsolete empty lines
- Add type check doc
- Rename inner function args to make pylint happy

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Support RBD namespaces (#30935)

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #31357 from votdev/improve_timediff_unittest

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cmake: detect gettid() presense

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #30534 from jiahuizeng/osd_dev_node

mgr/prometheus: assign a value to osd_dev_node when obj_store is not filestore or bluestore

Fix backend testing error due to attribute rename

`id` in Device is renamed to `path`.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #31431 from jschmid1/fix_cherrypy_imports

mgr/dashboard: cheroot moved into a separate project

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: Run python unit tests with tox in the mgr (#30364)

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: Remove compression mode unset in pool from (#31376)

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29944 from xxhdx1985126/wip-fix-blustore-fadvice_flags-leak

osd: copy ObjectOperation::BufferUpdate::Write::fadvise_flag to ceph::os::Transaction

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31333 from adamemerson/wip-who-warns-the-warnings

src: use un-deprecated version of aio_create_completion

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31416 from sseshasa/wip-41666-replicaSizeWarn

osd/OSDMap: Show health warning if a pool is configured with size 1

Reviewed-by: Sage Weil <sweil@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31475 from zdover23/wip-doc-community-calendar

doc: Added a link to Ceph Community Calendar.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc: Added a link to Ceph Community Calendar.

Co-Authored by: Neha Ojha <nojha@redhat.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #31501 into master

* refs/pull/31501/head:
ceph-mon: bind to public_addrv if set (before public_addr or public_network)
common/options: add public_addrv
common/options: fix ADDRVEC parsing
ceph-daemon: support --mon-addrv for deploy subcommand

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/crash: Calculate and add stack_sig to metadata

Signed-off-by: Dan Mick <dan.mick@redhat.com>

test/fio: bump to fio 3.15

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #30914 into master

* refs/pull/30914/head:
ceph: Add doc for deploying cephfs-nfs cluster using rook

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #31540 into master

* refs/pull/31540/head:
ceph-daemon: make ps1 a raw string

Reviewed-by: Sage Weil <sage@redhat.com>

osd/OSDMap.cc: don't output over/underfull messages to lderr

There can be cases where overfull and underfull(see example in
https://tracker.ceph.com/issues/42756) will be empty, which is not
necessarily an error. These error messages can end up spamming
the ceph-mgr log.

Fixes: https://tracker.ceph.com/issues/42756
Signed-off-by: Neha Ojha <nojha@redhat.com>

ceph-daemon: make ps1 a raw string

python3 interprets '\u' as a unicode escape

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #31024 into master

* refs/pull/31024/head:
doc: improve in mount.ceph man page
doc: describe mds_namespace option in mount.ceph man page

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30911 into master

* refs/pull/30911/head:
doc/ceph-fuse: describe -n option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #30657 into master

* refs/pull/30657/head:
test_cephfs_shell: pass arg to "help" in test_help
cephfs-shell: set exit_code when an error occurs
cephfs-shell: print traceback only when debug switch is on
cephfs-shell: catch libcephfs.Error only in onecmd()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31452 into master

* refs/pull/31452/head:
Add chown ceph:ceph for /tmp/ceph.mon.keyring

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/orchestrator_cli: sort by service name

This is more consistent when applied across hosts.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orchestrator_cli: collapse TYPE and ID columns

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31538 into master

* refs/pull/31538/head:
ceph-daemon: do not fetch daemon detail when removing a cluster
ceph-daemon: ls: add --no-detail

Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph-daemon: do not fetch daemon detail when removing a cluster

It's slow and not needed.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: ls: add --no-detail

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: add `--base-dir` arg to `adopt` command

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #31387 into master

* refs/pull/31387/head:
src/CMakeLists: remove leading 'v' from 'git describe' version

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31524 into master

* refs/pull/31524/head:
mgr/ssh: Fix various Python issues.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

mgr/dashboard: Improve position of MDS chart tooltip (#31368)

mgr/dashboard: Improve position of MDS chart tooltip

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #31512 into master

* refs/pull/31512/head:
mgr/ssh: allow passing LV to 'orchestrator osd create'

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: Support RBD namespaces

Fixes: https://tracker.ceph.com/issues/25125
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Fix data point alignment in MDS counters chart (#31288)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31519 from votdev/fix_ssh_vagrant

mgr/ssh: Install SSH public key in Vagrantfile box fails

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/ssh: Fix various Python issues.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: RBD tests must use pools with power-of-two pg_n… (#31490)

mgr/dashboard: RBD tests must use pools with power-of-two pg_num

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>