cephadm: consider stdout to get container version

Fixes: https://tracker.ceph.com/issues/57558
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #48149 from batrick/i57586

tools: allow spaces in dentries for first-damage.sh

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48057 from athanatos/sjust/wip-io-hang

crimson/osd/client_request: use fresh tracking_events/handle instances on requeue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd/client_request: use fresh tracking_events/handle instances on requeue

See instance_handle_t explanation in client_request.h

Fixes: https://tracker.ceph.com/issues/57494
Fixes: https://tracker.ceph.com/issues/57495
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #48059 from athanatos/sjust/wip-57506

crimson: fix for pgs stuck in +wait along with some debug improvements

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48091 from rzarzynski/wip-crimson-pgadvmap-complete-in-order

crimson/osd: PGAdvanceMap ops complete in order

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48134 from rhcs-dashboard/silence-fix

mgr/dashboard: use service call instead of form component

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

tools: allow spaces in dentries for first-damage.sh

Fixes: https://tracker.ceph.com/issues/57586
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: test first damage with dentry containing spaces

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #45369 from jyha200/wip-samplededup

ceph_dedup_tool: adding sample-dedup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #41855 from p-se/compile-cephadm-1

cephadm: compile cephadm

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47873 from pdvian/wip-log-last

mon/LogMonitor: Fix log last

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #47852 from Matan-B/wip-matanb-radosmodel

test/osd/RadosModel: Improve error debugging

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #47841 from Matan-B/wip-matanb-rados-striper

SimpleRADOSStriper: Avoid moving bufferlists by using deque in read()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #47774 from vumrao/wip-vumrao-57271

blk/KernelDevice: Modify the rotational and discard check log message

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #48156 from cbodley/wip-rgw-unused-throttles

rgw: remove unused AioThrottles in rgw_op.cc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #46321 from jmolmo/issue_55605

mgr/rook: fix error when trying to get the list of nfs services

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #46961 from tcoldrick-bb/balancer-weekday

mgr/balancer: Disallow 7 as {begin,end}_weekday and 2400 as {begin,end}_time

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46091 from pdvian/wip-bluestore-error

os/bluestore: Log error to stderr if fail to decode label

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #43791 from rzarzynski/wip-net-perfctrs-for-encrypted-traffic

auth, msg/async: add perf counters for encrypted traffic

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #48166 from rhcs-dashboard/fix-empty-pid-path-case

exporter: don't skip loop if pid path is empty

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: don't skip loop if pid path is empty

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Fixes issue when pid file config comes empty from config dump which prevents to add metrics. Also get process metrics only if
pid_path isn't empty.

Merge pull request #47986 from lmgdlmgd/patch-3

osd: set per_pool_stats true when OSD has no PG

Reviewed-by: Igor Fedotov <ifedotov@croit.io>

Merge pull request #48139 from ktdreyer/win-simplify-openssl

win32: skip installing openssl man pages

mgr/dashboard: use service call instead of form component

For creating the silence from the notification sidebar, instead of using
the silence form which will require initializing the whole component on
the landing page, we can just call the prometheus service and pass on
the required data to the service call. This will fix showing the
`Prometheus not configured` error everytime we visit the landing page when
the prometheus is not configured

Fixes: https://tracker.ceph.com/issues/57576
Signed-off-by: Nizamudeen A <nia@redhat.com>

ceph_dedup_tool: fix threshold comparison

If threadhold count is 2, existing code always returns true
even though reference count is 1 actually  because it additionally
adds one via ++.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

ceph_dedup_tool: make SampleDedupWorkerThread::chunk_size const

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupWorkerThread::duplicable_chunks

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: minor code simplifications

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unimplemented mark_non_dedup method signature

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: fix log lines to use std::endl, update object skip message

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupWorkerThread::do_object_dedup

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: make SampleDedupGlobal::sampling_rate const

Clarifies why it doesn't need to be protected by a lock.

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupGlobal::object_dedup_threshold

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: fix unnecessarily complicated bool returns

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: SampleDedupWorkerThread can simply inherit from Thread

SampleDedupWorkerThread didn't actually use any of the facilities on
CrawlerThread and indeed left several of them uninitialized.

Also allows us to remove the CrawlerThread constructor which left
several members uninitialized.

Signed-off-by: Samuel Just <sjust@redhat.com>

qa: remove unused values in deduplication.py

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

qa: add validation stage for deduplication.py

To validate sample-dedup actually works, validate() runs
separated thread from sample-dedup and verifies
two following things.
1. check sample-dedup starts properly.
2. check references of all the chunk objects' in chunk tier
   exists in designated base pool.
This routune repeats for max_valication_cnt times while
sample-dedup is running. If it doesn't raise any fail while the loop,
we can pretend sample-dedup works accurately.
If not, assert() will stop this test.

In case that a reference of chunk object doesn't exist in base pool,
validate() gives a second chance after repairing it (chunk-repair op)
to deal with false-positive reference inconsistency.

Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>

src/tools/ceph_dedup_tool: fixes to daemonize the process correctly

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

qa: add deduplication test under RGW workload (s3)

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #48157 from zdover23/wip-doc-2022-09-18-rados-network-config-prompt-correction

doc/rados: update prompts in network-config-ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: update prompts in network-config-ref

This PR adds unselectable prompts to network-config-
ref.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw: remove unused AioThrottles in rgw_op.cc

throttle creation was moved to RadosStore::get_atomic_writer(),
RadosStore::get_append_writer(), and RadosMultipartUpload::get_writer()

these instances in rgw_op.cc were unused

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47849 from torchiaf/fix/37327

mgr/dashboard: Add details to the modal which displays the `safe-to-d…

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #47958 from rhcs-dashboard/edit-host-labels-fix

mgr/dashboard: fix Expected to find element: `cd-modal .badge but never found it

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

win32: skip installing openssl man pages

This speeds up the build a bit and makes the logs easier to read.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #45148 from guits/revert-ceph-crash-changes

Revert "src/ceph-crash.in: remove unused frame in handler()"

Reviewed-by: Teoman ONAY <tonay@redhat.com>

Merge pull request #47965 from cbodley/wip-rgw-drain-cb-capture-error

Revert "rgw: add coroutine drain callback capture_error()"

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

mgr/dashboard: Add details to the modal which displays the `safe-to-destroy` result

- Add warnings type information in the case of the OSDs are not safe to destroy
- Add info type information in the case of the OSDs are safe to destroy

Fixes: https://tracker.ceph.com/issues/37327
Signed-off-by: Francesco Torchia <francesco.torchia@suse.com>

Merge pull request #47878 from xxhdx1985126/wip-pin_set-cpu

crimson/os/seastore/btree: reduce the avg number of queries of pin sets' parent/child lookup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #47090 from lxbsz/wip-56553

client: do not uninline data for read

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #48037 from guits/fix_57085

ceph-volume: fix inventory with device arg

Merge pull request #48117 from guits/fix-cv-func-test-filestore-create

ceph-volume/tests: fix lvm centos8-filestore-create job

Merge pull request #48054 from yuvalif/wip-yuval-fix-56572

tests: remove pubsub tests from multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kalpesh Pandya <kapandya@redhat.com>

Merge pull request #47516 from nmshelke/fix-56632

qa: ignore disk quota exceeded failure in test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

ceph-volume/tests: fix lvm centos8-filestore-create job

/dev/vdc1 can't be zapped if it still holds an lv mapper.
let's use --destroy in the lvm zap command in order to remove
the held lv mapper before zapping the partition and recreate the partition after.

Fixes: https://tracker.ceph.com/issues/57553
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

crimson/os/seastore/btree: reduce the avg number of queries of pin sets' parent/child lookup

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #47966 from cyx1231st/wip-seastore-move-gcprocess

crimson/os/seastore: move AsyncCleaner::GCProcess to EPM::BackgroundProcess

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #48105 from athanatos/sjust/wip-57530

crimson/.../node.cc: fix Node initialization argument passing

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/.../node.cc: fix Node initialization argument passing

Argument evaluation order is unspecified, clang seems to be choosing
to execute the std::unique_ptr move constructor before the call to
get(), causing the first argument to be passed as null.

https://tracker.ceph.com/issues/57530
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #48002 from nSedrickm/accessibility-documentation

doc/dashboard: update documentation with accessibility development pr…

Merge pull request #46723 from nSedrickm/dashboard-health-accessibility-update

mgr/dashboard: Improve accessibility for dashboard health page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #47668 from NitzanMordhai/wip-nitzan-test-rados-watch-notify-callbacks

pybind/rados: notify callback reconnect

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #47937 from adk3798/tp-add-rm-settings-docs

doc/cephadm: fix tuned-profile add/rm-setting syntax example

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47709 from zalsader/add-daos

rgw: add DAOS SAL implementation

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #48066 from yuvalif/wip-yuval-fix-57518

rgw/lua: create lua background manager before the frontends

Reviwed-by: dang, cbodley, mattbenjamin

Revert "rgw: add coroutine drain callback capture_error()"

This reverts commit d75ea8757d96a82df692a9476ed40b3206cba47e.

Fixes: https://tracker.ceph.com/issues/57400
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43655 from liewegas/auth-rotate

mon,auth,cephadm: support auth key rotation

Merge pull request #47967 from yangxiaolaing/cephadm_doc

doc/cephadm: Add creation method of OSD

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

crimson/osd: improve debugs in PGAdvanceMap

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: PGAdvanceMap ops complete in order

For the analysis refer to the bug tracker.

Dropping the `handle.exit()` looks like the simplest solution,
far simpler than e.g. introducing extra future-promise.

Fixes: https://tracker.ceph.com/issues/57536
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: use build.py in cmake configuration

This change replaces the call to build.sh with a call to build.py.

It also makes the build.py script a dependency to the cmake custom to
ensure we rebuild the cephadm binary if build.py changes.  Since, if one
changes the build script used to make the binary we should rebuild the
output as the new script may indeed change the intended output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add a pythonic build.py for constructing the cephadm binary

As discussed in a ceph orch meeting, the build.sh script was deemed
"unpythonic". This script is a first attempt to do it more pythonic
with fewer explicit version checks.

This change minimizes the existing build.sh to simply call build.py.
We can completely eliminate build.sh at a later time if desired.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/tasks/cephadm: conditionally pull cephadm binary from chacra

When a specific (older) version of cephadm is needed for a test, pull
it from chacra. If we need to bootstrap an older version of ceph, we
pull an older "single-script" binary using the older method.

Co-authored-by: Michael Fritch <mfritch@suse.com>
Co-authored-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/tasks: add chacra util

to query shaman and pull a binary from chacra

Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

behave_test: compile cephadm

build the compiled cephadm binary during the behave test

Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: compile cephadm

build the compiled cephadm binary during the e2e test

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: compile cephadm

use cephadm.py when mounting via the `--dev_share_folder` option

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm: add `--no-pager`

avoid user interaction with the systemd pager

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_repos: add zypper

cephadm adds partial enablement for zypper repos

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_repos: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_adoption: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_adoption: remove python kludge

we are no longer testing against py2 as of 225cbd8d790602b30050e0e32ecffd400ec17731

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm: compile cephadm

build the compiled cephadm binary during the workunit test

Signed-off-by: Michael Fritch <mfritch@suse.com>

cpatch: compile the cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

cstart.sh: compile cephadm

update cstart/ckill scripts to use the compiled cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

stop.sh: compile cephadm

use the path to the compiled cephadm binary file

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: allow execution from a relative path

let the build script run from a relative path (e.g. build or qa dir)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: zipapp module requires Python35 or later

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: locate path to the python3 binary

fixes incorrect shebang in the compiled zipfile (`#!python3`)

Signed-off-by: Michael Fritch <mfritch@suse.com>

rpm, debian: use cmake to install the cephadm binary

Signed-off-by: Michael Fritch <mfritch@suse.com>

cmake: add cephadm to vstart target

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: Only compress cephadm on Python >= 3.7

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

qa/tasks: Drop downloading cephadm from the git source tree

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: compile cephadm

Compiling cephadm into an single binary offers the ability to split the
code from cephadm up into several files/modules. A side effect of having
a `cephadm.py` file (which, by renaming to cephadm.py has already become
a module) is that it can properly be imported and static type checking
inside `src/cephadm/tests/test_cephadm.py` is not an issue anymore.

Next steps would to split up cephadm into several modules and adapting
the test_cephadm.py import mechanism, as well as fixing the typing
issues, which are revealed by using a proper important statement.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47521 from adk3798/iscsi-ip-list

mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47640 from adk3798/no-cgroups

mgr/cephadm: make setting --cgroups=split configurable

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47936 from adk3798/tuned-profile-dashes

mgr/cephadm: fix tuned profiles getting removed if name has dashes

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47786 from rkachach/fix_issue_57192

mgr/cephadm: validating tuned profile specification

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48008 from adk3798/docs-extra-container-mounts

doc/cephadm: document recommended syntax for mounting files with ECA

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>