Merge pull request #35033 from badone/wip-test-the-right-rocksdb-version

qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35032 from badone/wip-update-rocksdb-to-v6.8.1

rocksdb: Updated to v6.8.1

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #35111 from agayev/hm-smr

os/bluestore: Enable writing data to HM-SMR hard drives.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35097 from matthewoliver/cephadm_iscsi_api_user_password

cephadm: Make ceph-iscsi api user and password mandatory

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35079 from liewegas/wip-dedup-tool

ceph-dedup-tool: add new FastCDC chunker, and make estimate test a range of chunk sizes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #35198 from varshar16/wip-cephadm-module-error

qa/tasks/mgr: Don't disable cephadm module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35128 from yuvalif/fix-amqp-urls-with-vhosts

rgw/url: fix amqp urls with vhosts

common/rabin: remove

The implementation is buggy, and slower than FastCDC.

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: default to fastcdc

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: better behavior if TARGET_WINDOW_BITS==0

Skip the middle scan call entirely in that case.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: add some comments

Map terms back to those in the paper.

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: make calc_chunks const

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: take const bufferlist

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: use bufferlist iterator

More complex, but we avoid a buffer copy/rebuild if it is non-contiguous.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: simplify initial fingerprint calc

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: refactor scan into a helper

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: generate fragmented buffers

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: add test with a specific chunking result

Canary for alg changes

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: show object count explosion too

Also show dedup ratio as dedup_size/original_size (1.0 = no dedup,
.9 = 10%, etc.)

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: show estimate progress

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: test a range of chunk sizes at once

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: add --max-seconds

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: timeout -> report_period

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: use CDC class for chunking

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: useful error messages

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: EstimateThread -> CrawlerThread

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: remove rabin-specific options

We don't want to fiddle with these.

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_cdc: adjust tests a bit

Seed random buffers!

Include average in histogram, and do the binning slightly differently.

Signed-off-by: Sage Weil <sage@newdream.net>

common/FixedCDC: add 'fixed' chunker

Signed-off-by: Sage Weil <sage@newdream.net>

common/FastCDC: initial implementation

Loosely based on the paper, but with a few changes.

Signed-off-by: Sage Weil <sage@newdream.net>

common/CDC: add abstract interface

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: warn on hash collision

Signed-off-by: Sage Weil <sage@newdream.net>

unittest_rabin_chunk: add unit test

This includes a "shift" test that shows that the rabin chunking misbehaves
when 5+ bytes are inserted at the front (many chunk boundaries shift!)

Signed-off-by: Sage Weil <sage@newdream.net>

common/rabin: add helper set_target_bits()

Sets key chunking parameters in one easy call.

Signed-off-by: Sage Weil <sage@newdream.net>

common/rabin: some cleanups

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: rename var

Signed-off-by: Sage Weil <sage@newdream.net>

ceph-dedup-tool: larger default op size

If we don't read the entire object the rabin calculation won't work.  Avoid
this by default by just using a sufficiently large chunk size.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #34902 from p-se/wip-pse-doc-prom-security

mgr/dashboard: document Prometheus' security model

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34863 from rhcs-dashboard/45310-add-grafana-rgw-sync-info

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34321 from jan--f/c-v-manage-lv_tags-bulk

ceph-volume: add and delete lvm tags in a single lvchange call.

Merge pull request #35235 from sebastian-philipp/qa-cephadm-mirror

qa/cephadm: Add local dockerhub mirror

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #34931 from trociny/wip-45072-2

rbd-mirror: wait for in-flight start/stop/restart

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35245 from ivancich/wip-remove-unused-func

rgw: remove unused function obj_force_ns

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/cephadm: Add local registry mirror

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

rgw: remove unused function obj_force_ns

Removing the function eliminates a compiler warning.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #35089 from trociny/wip-test-quiesce_watch_timeout

test/librbd: improve TestLibRBD.QuiesceWatchTimeout

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35209 from wjwithagen/wjw-fix-librbd-CephContext

librbd: fix compile error with CephContext declaration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35228 from changchengx/rwl

rbd/rwl: correct m_perfcounter access scope

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34294 from ishanrai05/copy-button

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34973 from liewegas/doc-pingan

doc/foundation: PingAn canceled general membership

Merge pull request #35172 from yuvalif/add-flags-to-rgw-operate

rgw/tools: add flags to rgw_rados_operate api

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35200 from rzarzynski/wip-crimson-ertr-assert_all

crimson: bring assert_all to errorator

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34079 from ishanrai05/login

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #34507 into master

* refs/pull/34507/head:
mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit

Reviewed-by: Zheng Yan <zyan@redhat.com>

mgr/dashboard: document Prometheus' security model

Fixes: https://tracker.ceph.com/issues/45377
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

rbd/rwl: correct m_perfcounter access scope

There's no m_perfcounter in SyncPoint. Its scope is in
ReplicatedWriteLog.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

rgw/url: fix commensts from review #34717

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269

src/rgw/rgw_url.cc, also cope with a vhost appended to a AMQP-URL

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269

crimson/common: discard_all::operator() returns void now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os/seastore: switch to crimson::ct_error::assert_all.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common: extend errorator with assert_all{}.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34953 from s0nea/wip-dashboard-telemetry-url

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35196 from tspmelo/wip-logout

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #35129 from smithfarm/wip-fix-PendingReleaseNotes-headings

doc/releases/octopus.rst: move still-pending release notes to official v15.2.0 release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

doc/releases/octopus: use fixed-width font for "ceph fs dump"

We are using fixed-width font to render commands elsewhere, so
let's do so in this release note, too - just for the sake of uniformity.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35142 from mgfritch/cephadm-validate-mds-fsname

mgr/orch: allow MDS with explicit naming

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34174 from theanalyst/doc/active-releases-oct

doc: move luminous to archived releases & octopus to active

Reviewed-by: Nathan Cutler <ncutler@suse.com>

librbd: fix compile error with CephContext declaration

Clang complains:
/home/jenkins/workspace/ceph-master/src/librbd/plugin/Types.h:9:1: error: declaration conflicts with target of using declaration already in scope
struct CephContext;
^
/home/jenkins/workspace/ceph-master/src/include/common_fwd.h:10:9: note: target of using declaration
class CephContext;
^
/home/jenkins/workspace/ceph-master/src/include/common_fwd.h:22:24: note: using declaration
using TOPNSPC::common::CephContext;
^

fixes: https://github.com/ceph/ceph/pull/35158
tracker: https://tracker.ceph.com/issues/45668
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #35219 from tchaikov/wip-crimson-cleanups

crimson/osd,test/crimson: avoid using variadic future and s/apply/invoke/

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

crimson/osd: avoid using variadic future

see also 3cee25ba41069e9e9b4855f99be20910759440ca

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: use invoke instead of apply

it's an oversight in 33af821baafbb2e9c4ebea2e377db665e210701f

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35204 from tchaikov/wip-tools-cephfs

tools/cephfs/MetaTool: print buffer::error::what()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35207 from tchaikov/wip-crimson-remove-vardiadict-future

crimson/osd: avoid using variadic future and cleanup

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: remove redundant std::move()

this should address following warning:

../src/crimson/osd/ops_executer.cc:821:6:   required from here
../src/crimson/osd/ops_executer.cc:818:37: warning: redundant move in return statement [-Wredundant-move]
  818 |             return std::move(entries);
      |                                     ^
../src/crimson/osd/ops_executer.cc:818:37: note: remove ‘std::move’ call

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: avoid using variadic future

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35205 from Huber-ming/zsm_doc

doc: replace https://ceph.com/ with http://download.ceph.com/

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc:replacing https://ceph.com/ by http://download.ceph.com/

Signed-off-by: Shengming Zhang <zhangsm01@inspur.com>

Merge pull request #35179 from myoungwon/wip-affiliation-mw

mailmap: update affiliation for Myoungwon Oh

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/cephfs/MetaTool: print buffer::error::what()

* buffer::error does not offer operator<<(ostream&, const
  buffer::error&). so in this change, err.what() is used instead.
* also add spaces to follow our coding style convention.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #34755 into master

* refs/pull/34755/head:
tool: add function of show/amend fnode

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #35090 into master

* refs/pull/35090/head:
mgr/volumes: Fix subvolume create idempotency

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #35063 into master

* refs/pull/35063/head:
cephfs: allow pool names with '-' and '.'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #34975 into master

* refs/pull/34975/head:
systemd: lock down more privileges

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #34757 into master

* refs/pull/34757/head:
mds: take xlock in the order requests start locking

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #35143 into master

* refs/pull/35143/head:
mgr/status: 'ceph fs status' output add dirs and caps

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35026 from changchengx/rwl

librbd: track per syncpoint's logentry number with perf counter

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #35109 into master

* refs/pull/35109/head:
qa: use py3 compat list from range

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32601 from adamemerson/wip-objection-triumphant

Asynchronous work on objecter and asynchronous RADOS library.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

os/bluestore: Enable writing data to HM-SMR hard drives.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

mgr/dashboard: Add button to copy the bootstrap token into the clipboard

Fixes issue: https://tracker.ceph.com/issues/44681

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

mgr/dashboard: Use Web Clipboard API to copy to clipboard

Replace the deprecated Document.execCommand with Web Clipboard API.

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

Merge pull request #35020 from yuvalif/cls_2pc_q_async_api

cls/2pc_queue: add async API to read operations

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34505 from ktdreyer/doc-forwarded-https

doc/rgw: warn about "trust forwarded https" security

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35158 from dillaman/wip-45137

librbd: initial implementation of plugin support

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/tasks/mgr: Don't disable cephadm module

Fixes: https://tracker.ceph.com/issues/45596
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/dashboard: grafana panels for rgw multisite sync performance

* RGW sync perf. counters are now exposed through grafana panels.
* Sync Performance tab is only shown if rgw realm is detected.
* Prometheus module: added metrics suitable for prometheus consumption (from existing ones, not replacing for backward compatibility).

Fixes: https://tracker.ceph.com/issues/45310
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard: Fix redirect after changing password

After the changes made to subscriptions, redirecting to the dummy logout page
no longer does anything.

So we removed the logout route and now redirect directly to the login page.

Fixes: https://tracker.ceph.com/issues/45651
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/volumes: Fix subvolume create idempotency

After subvolume is created, it can be resized
using subvolume create command. But it was
broken and the same is fixed.

Fixes: https://tracker.ceph.com/issues/45398
Signed-off-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #33964 from rosinL/openss_engine

common, rgw: add openssl engine support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #33237 from aclamk/wip-bluefs-list-files

os/bluestore/bluefs: add inspection of bluefs files

Reviewed-by: Igor Fedotov <ifedotov@suse.com>