qa/tasks/cephadm: Add check fox already set image

Adding a check for already bootstrapped clusters where the image is
already set to avoid overriding it.

Signed-off-by: Georgios Kyratsas <gkyratsas@suse.com>

Merge pull request #35222 from smithfarm/wip-45606

osd: make "missing incremental map" a debug log message

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35194 from bk201/wip-45625

mgr/cephadm: config service_url in Dashboard for Prometheus and Alert…

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33872 from majianpeng/osd-remove-duplicated-assert

osd/OSD: remove useless ceph_assert.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35221 from tchaikov/wip-22052

qa/valgrind.supp: less specific when suppressing issues/22052

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34835 from jschmid1/osdspec_affinity

osd: add --osdspec-affinity flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35259 from tchaikov/wip-fmt-mingw

fmt: pickup change for adding MinGW support

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr: include retval in MonCommandFailed exception

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

mgr/cephadm: config service_url in Dashboard for Prometheus and AlertManager

Calling Dashboard's CLI to set service URLs after deploying Prometheus
and AlertManager Daemons.

Fixes: https://tracker.ceph.com/issues/45625
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #35197 into master

* refs/pull/35197/head:
os/Transaction: do not use anonymous aggregate with member having ctor
include/interval_set: no need to check if an unsigned greater or equal to 0
include/ceph_fs: do not use anonymous aggregate with member having ctor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35229 from ricardoasmarques/wip-45696

cephadm: Validate bootstrap "--dashboard-{key|crt}" path

Reviewed-by: Daniel-Pivonka <dpivonka@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #35106 from p-se/wip-pse-cephadm-custom-monitoring-images

mgr/cephadm: allow custom images for grafana, prometheus, alertmanager and node_exporter

Merge pull request #35159 from batrick/i45632

pybind/mgr/cephadm: remove mds auth cap for nfs recovery db cred

Merge pull request #35266 from jschmid1/fix_previews_for_multimatch

mgr/cephadm: fix host matching when multiple hosts match

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35195 from ricardoasmarques/allow-specify-ssh

cephadm: Allow users to provide ssh keys during bootstrap

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #35273 from sebastian-philipp/qa-cephadm-mirror-ubuntu

qa/cephadm: Docker doesn't ship a registries.conf

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #35267 into master

* refs/pull/35267/head:
vstart_runner: set mounted to True at the end of mount()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #34974 from dungdm93/patch-1

pybind/ceph_volume_client: Fix PEP-8 SyntaxWarning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35012 from tchaikov/wip-45147

qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

Reviewed-by: Yuri Weinstein <yuriw@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35033 from badone/wip-test-the-right-rocksdb-version

qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35032 from badone/wip-update-rocksdb-to-v6.8.1

rocksdb: Updated to v6.8.1

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #35111 from agayev/hm-smr

os/bluestore: Enable writing data to HM-SMR hard drives.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35097 from matthewoliver/cephadm_iscsi_api_user_password

cephadm: Make ceph-iscsi api user and password mandatory

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35079 from liewegas/wip-dedup-tool

ceph-dedup-tool: add new FastCDC chunker, and make estimate test a range of chunk sizes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #35198 from varshar16/wip-cephadm-module-error

qa/tasks/mgr: Don't disable cephadm module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35128 from yuvalif/fix-amqp-urls-with-vhosts

rgw/url: fix amqp urls with vhosts

common/rabin: remove

The implementation is buggy, and slower than FastCDC.

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: default to fastcdc

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: better behavior if TARGET_WINDOW_BITS==0

Skip the middle scan call entirely in that case.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: add some comments

Map terms back to those in the paper.

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: make calc_chunks const

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: take const bufferlist

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: use bufferlist iterator

More complex, but we avoid a buffer copy/rebuild if it is non-contiguous.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: simplify initial fingerprint calc

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: refactor scan into a helper

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: generate fragmented buffers

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: add test with a specific chunking result

Canary for alg changes

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: show object count explosion too

Also show dedup ratio as dedup_size/original_size (1.0 = no dedup,
.9 = 10%, etc.)

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: show estimate progress

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: test a range of chunk sizes at once

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: add --max-seconds

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: timeout -> report_period

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: use CDC class for chunking

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: useful error messages

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: EstimateThread -> CrawlerThread

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: remove rabin-specific options

We don't want to fiddle with these.

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: adjust tests a bit

Seed random buffers!

Include average in histogram, and do the binning slightly differently.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FixedCDC: add 'fixed' chunker

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: initial implementation

Loosely based on the paper, but with a few changes.

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: add abstract interface

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: warn on hash collision

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_rabin_chunk: add unit test

This includes a "shift" test that shows that the rabin chunking misbehaves
when 5+ bytes are inserted at the front (many chunk boundaries shift!)

Signed-off-by: Sage Weil <sage@newdream.net>

common/rabin: add helper set_target_bits()

Sets key chunking parameters in one easy call.

Signed-off-by: Sage Weil <sage@newdream.net>

common/rabin: some cleanups

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: rename var

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: larger default op size

If we don't read the entire object the rabin calculation won't work.  Avoid
this by default by just using a sufficiently large chunk size.

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: fix host matching when multiple hosts match for in osd
previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>

qa/cephadm: Docker doesn't ship a registries.conf

I guess not using the mirror for docker based tests is ok for now.

For adding docker support, we need to:

1. change the docker config
2. restart the docker daemon

Note that Docker's config only supports to mirror the docker.io registry.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

cephadm: Alow users to provide ssh config during bootstrap

Custom ssh config can be provided by using the '--ssh-config' option

Signed-off-by: Ricardo Marques <rimarques@suse.com>

cephadm: Allow users to provide ssh keys during bootstrap

Fixes: https://tracker.ceph.com/issues/45629
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #34902 from p-se/wip-pse-doc-prom-security

mgr/dashboard: document Prometheus' security model

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

vstart_runner: set mounted to True at the end of mount()

Not doing so prevents cleanup methods from running.

Fixes: https://tracker.ceph.com/issues/45723
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34863 from rhcs-dashboard/45310-add-grafana-rgw-sync-info

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34321 from jan--f/c-v-manage-lv_tags-bulk

ceph-volume: add and delete lvm tags in a single lvchange call.

fmt: pickup change for adding MinGW support

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/Transaction: do not use anonymous aggregate with member having ctor

use a single `hint` to replace `hint_type` and `alloc_hint_flags` in
the anonymous union.

this helps to address the compile error reported by GCC-10

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/interval_set: no need to check if an unsigned greater or equal to 0

this silences the warning of

../src/include/interval_set.h:512:23: warning: comparison of unsigned expression in ‘>= 0’ is always true [-Wtype-limits]
  512 |     ceph_assert(_size >= 0);

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35235 from sebastian-philipp/qa-cephadm-mirror

qa/cephadm: Add local dockerhub mirror

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #34931 from trociny/wip-45072-2

rbd-mirror: wait for in-flight start/stop/restart

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35245 from ivancich/wip-remove-unused-func

rgw: remove unused function obj_force_ns

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/cephadm: Add local registry mirror

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

rgw: remove unused function obj_force_ns

Removing the function eliminates a compiler warning.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #35089 from trociny/wip-test-quiesce_watch_timeout

test/librbd: improve TestLibRBD.QuiesceWatchTimeout

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35209 from wjwithagen/wjw-fix-librbd-CephContext

librbd: fix compile error with CephContext declaration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35228 from changchengx/rwl

rbd/rwl: correct m_perfcounter access scope

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34294 from ishanrai05/copy-button

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34973 from liewegas/doc-pingan

doc/foundation: PingAn canceled general membership

Merge pull request #35172 from yuvalif/add-flags-to-rgw-operate

rgw/tools: add flags to rgw_rados_operate api

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/cephadm: remove unused imports

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: reformat according to pep8

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: log all args not only in root mode

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: remove unused variable

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: add types to function parameters

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: custom images for monitoring components

Namely,
- Grafana
- Prometheus
- Alertmanager and
- Node exporter.

Fixes: https://tracker.ceph.com/issues/45463
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #35200 from rzarzynski/wip-crimson-ertr-assert_all

crimson: bring assert_all to errorator

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34079 from ishanrai05/login

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #34507 into master

* refs/pull/34507/head:
mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit

Reviewed-by: Zheng Yan <zyan@redhat.com>

mgr/dashboard: document Prometheus' security model

Fixes: https://tracker.ceph.com/issues/45377
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

rbd/rwl: correct m_perfcounter access scope

There's no m_perfcounter in SyncPoint. Its scope is in
ReplicatedWriteLog.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

rgw/url: fix commensts from review #34717

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269

src/rgw/rgw_url.cc, also cope with a vhost appended to a AMQP-URL

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269

crimson/common: discard_all::operator() returns void now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os/seastore: switch to crimson::ct_error::assert_all.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common: extend errorator with assert_all{}.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34953 from s0nea/wip-dashboard-telemetry-url

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35196 from tspmelo/wip-logout

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cephadm: Validate bootstrap "--dashboard-{key|crt}" path

Fixes: https://tracker.ceph.com/issues/45696
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #35129 from smithfarm/wip-fix-PendingReleaseNotes-headings

doc/releases/octopus.rst: move still-pending release notes to official v15.2.0 release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

doc/releases/octopus: use fixed-width font for "ceph fs dump"

We are using fixed-width font to render commands elsewhere, so
let's do so in this release note, too - just for the sake of uniformity.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35142 from mgfritch/cephadm-validate-mds-fsname

mgr/orch: allow MDS with explicit naming

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>