RGW - Re-do admin API split

The mgr instantiation was re-added in the build re-org.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #47131 from aaryanporwal/vrt-doc-update

doc/dev: mgr/dashboard: Replace broken link

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #48217 from cyx1231st/wip-crimson-buffer

crimson: create buffer from temporary_buffer with foreign-ptr by default

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48235 from zdover23/wip-doc-2022-09-25-dev-guide-basic-workflow-fork-note-formatting

doc/dev: improve presentation of note (git remote)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: improve presentation of note (git remote)

This commit corrects the formatting of a note that previously
did not make console output and commands as clear as this commit
makes them.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #47767 from batrick/i57249

mds: correct storage of multiple damaged dentries in DamageTable

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #48148 from ktdreyer/win-remove-curl

win32: remove libcurl

Merge pull request #47080 from dparmar18/wip-dparmar-MDS-56522

mds/Server: Do not abort MDS on unknown messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

crimson: create buffer from temporary_buffer with foreign-ptr by default

temporary_buffer is internally shareable with a thread-unsafe
ref-counter, we need to make sure it is released in the same core where
it is constructed.

Users that need the extra efficiency can swap to create_local as needed.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #47265 from s0nea/wip-dashboard-redirect-fqdn

mgr/dashboard: add option to resolve ip addr

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #48198 from mgfritch/cephadm-pytest-patch-logger

cephadm: patch the cephadm.logger class

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #48095 from ivancich/wip-unify-logic

rgw: fix bool/int logic error when calling get_obj_head_ioctx

Merge pull request #47756 from dparmar18/wip-dparmar-cephadm-after-revert

pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #48118 from s0nea/container-version-fix

cephadm: consider stdout to get container version

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #48120 from mgfritch/cephadm-haproxy-nonlocal-bind

cephadm: add `ip_nonlocal_bind` to haproxy deployment

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48184 from SUSE/wip-fix-57609

python-common: Add 'KB' to supported suffixes in SizeMatcher

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #48200 from guits/fix-57627

ceph-volume: fix regression in activate

Merge pull request #48185 from zhscn/wip-dtype

crimson/os/seastore: introduce backend_type_t

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: make device according to device type

Signed-off-by: Zhang Song <zhangsong325@gmail.com>

crimson/os/seastore: introduce get_backend_type to Device

Signed-off-by: Zhang Song <zhangsong325@gmail.com>

crimson/os/seastore: make can_delay_allocation always return true

Signed-off-by: Zhang Song <zhangsong325@gmail.com>

crimson/os/seastore: rename about device_type_t and journal_type_t

Signed-off-by: Zhang Song <zhangsong325@gmail.com>

crimson/os/seastore: introduce backend_type_t

Signed-off-by: Zhang Song <zhangsong325@gmail.com>

Merge pull request #48012 from Huber-ming/slave_requestPayment

rgw: set requestPayment in slave zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48021 from soumyakoduri/wip-multisite-crash

rgw: Drain async_processor request queue during shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

ceph-volume: fix regression in activate

bea9f4b643c introduced a regression that makes the activate process
take a very long time to complete.
`_get_bluestore_info()` which calls `ceph-bluestore-tool` binary via
subprocess is called in an exponential way while this is not needed.

Fixes: https://tracker.ceph.com/issues/57627
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

cephadm: denote any mocked params as internal

prefix the mock patch params with a `_` char for consistancy with other
tests cases within the suite

Fixes: Fixes: https://tracker.ceph.com/issues/57621
Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: patch the `cephadm.logger` class

Patch the logger class instead of globally mocking the class from within
the loaded source file. This was inadvertently allowing for the entire
test run to succeed, while a single run of a test case would fail due to
the missing mock.

For example:
    `tox -e py3 tests/test_cephadm.py::TestShell::test_fsid`

Fixes: Fixes: https://tracker.ceph.com/issues/57621
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #47766 from cbodley/wip-qa-rgw-tempest-meta

qa/rgw/tempest: blocklist failing tests for removal of container metadata

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #48193 from rhcs-dashboard/fix-stoi

exporter: avoid stoi for empty pid_str

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #47900 from Matan-B/wip-matanb-librados-20

examples/librados: Compile with c++20 on Ubuntu

cephadm: add `ip_nonlocal_bind` to haproxy deployment

standalone haproxy deployments require the
`net.ipv4.ip_nonlocal_bind` sysctl setting

Fixes: https://tracker.ceph.com/issues/57563
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #48191 from phlogistonjohn/jjm-cephadm-tox-coverage

cephadm: add a tox env for generating a coverage report

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47494 from rhcs-dashboard/pgs-repaired

mgr/prometheus: expose repaired pgs metrics

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #48190 from ronen-fr/wip-rf-replica-init

osd/scrub: make on_replica_init() idempotent again

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>

exporter: avoid stoi for empty pid_str

Fixes: https://tracker.ceph.com/issues/57619
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #48121 from rzarzynski/wip-crimson-pgadvmap-critical-from

crimson/osd: initialize PGAdvanceMap::from inside critical section

Reviewed-by: Samuel Just <sjust@redhat.com>

cephadm: add a tox env for generating a coverage report

This "dormant" target is here for developers to run locally and generate
a HTML coverage report to review.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #48131 from adamemerson/wip-no-fno-new-ttp-matching

build: Remove -fno-new-ttp-matchingg flag

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

osd/scrub: make on_replica_init() idempotent again

on_replica_init() might be called twice during replica scrub
initiation. Thus, it was designed to cause no issue if called
an extra time. That was broken when the scrubber-backend code
was introduced.

Fixes: https://tracker.ceph.com/issues/57616
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #48172 from mchangir/doc-add-note-about-cephfs-mirroring-paths

doc/cephfs: add note about cephfs-mirroring paths

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/cephfs: add note about cephfs-mirroring paths

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephadm: consider stdout to get container version

Fixes: https://tracker.ceph.com/issues/57558
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

python-common: Add 'KB' to supported suffixes in SizeMatcher

Fixes: https://tracker.ceph.com/issues/57609
Signed-off-by: Tim Serong <tserong@suse.com>

Merge pull request #48149 from batrick/i57586

tools: allow spaces in dentries for first-damage.sh

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48057 from athanatos/sjust/wip-io-hang

crimson/osd/client_request: use fresh tracking_events/handle instances on requeue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd/client_request: use fresh tracking_events/handle instances on requeue

See instance_handle_t explanation in client_request.h

Fixes: https://tracker.ceph.com/issues/57494
Fixes: https://tracker.ceph.com/issues/57495
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #48059 from athanatos/sjust/wip-57506

crimson: fix for pgs stuck in +wait along with some debug improvements

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48091 from rzarzynski/wip-crimson-pgadvmap-complete-in-order

crimson/osd: PGAdvanceMap ops complete in order

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48134 from rhcs-dashboard/silence-fix

mgr/dashboard: use service call instead of form component

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

tools: allow spaces in dentries for first-damage.sh

Fixes: https://tracker.ceph.com/issues/57586
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: test first damage with dentry containing spaces

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #45369 from jyha200/wip-samplededup

ceph_dedup_tool: adding sample-dedup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

mgr/prometheus: expose num objects repaired in pool

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41855 from p-se/compile-cephadm-1

cephadm: compile cephadm

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47873 from pdvian/wip-log-last

mon/LogMonitor: Fix log last

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #47852 from Matan-B/wip-matanb-radosmodel

test/osd/RadosModel: Improve error debugging

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #47841 from Matan-B/wip-matanb-rados-striper

SimpleRADOSStriper: Avoid moving bufferlists by using deque in read()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #47774 from vumrao/wip-vumrao-57271

blk/KernelDevice: Modify the rotational and discard check log message

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #48156 from cbodley/wip-rgw-unused-throttles

rgw: remove unused AioThrottles in rgw_op.cc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #46321 from jmolmo/issue_55605

mgr/rook: fix error when trying to get the list of nfs services

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #46961 from tcoldrick-bb/balancer-weekday

mgr/balancer: Disallow 7 as {begin,end}_weekday and 2400 as {begin,end}_time

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46091 from pdvian/wip-bluestore-error

os/bluestore: Log error to stderr if fail to decode label

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #43791 from rzarzynski/wip-net-perfctrs-for-encrypted-traffic

auth, msg/async: add perf counters for encrypted traffic

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

examples: Compile with c++20

This change will apply to ubuntu only.
Fixes: https://tracker.ceph.com/issues/57163
Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #48166 from rhcs-dashboard/fix-empty-pid-path-case

exporter: don't skip loop if pid path is empty

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

exporter: don't skip loop if pid path is empty

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Fixes issue when pid file config comes empty from config dump which prevents to add metrics. Also get process metrics only if
pid_path isn't empty.

Merge pull request #47986 from lmgdlmgd/patch-3

osd: set per_pool_stats true when OSD has no PG

Reviewed-by: Igor Fedotov <ifedotov@croit.io>

Merge pull request #48139 from ktdreyer/win-simplify-openssl

win32: skip installing openssl man pages

mgr/dashboard: use service call instead of form component

For creating the silence from the notification sidebar, instead of using
the silence form which will require initializing the whole component on
the landing page, we can just call the prometheus service and pass on
the required data to the service call. This will fix showing the
`Prometheus not configured` error everytime we visit the landing page when
the prometheus is not configured

Fixes: https://tracker.ceph.com/issues/57576
Signed-off-by: Nizamudeen A <nia@redhat.com>

ceph_dedup_tool: fix threshold comparison

If threadhold count is 2, existing code always returns true
even though reference count is 1 actually  because it additionally
adds one via ++.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

ceph_dedup_tool: make SampleDedupWorkerThread::chunk_size const

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupWorkerThread::duplicable_chunks

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: minor code simplifications

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unimplemented mark_non_dedup method signature

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: fix log lines to use std::endl, update object skip message

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupWorkerThread::do_object_dedup

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: make SampleDedupGlobal::sampling_rate const

Clarifies why it doesn't need to be protected by a lock.

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: remove unused SampleDedupGlobal::object_dedup_threshold

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: fix unnecessarily complicated bool returns

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_dedup_tool: SampleDedupWorkerThread can simply inherit from Thread

SampleDedupWorkerThread didn't actually use any of the facilities on
CrawlerThread and indeed left several of them uninitialized.

Also allows us to remove the CrawlerThread constructor which left
several members uninitialized.

Signed-off-by: Samuel Just <sjust@redhat.com>

qa: remove unused values in deduplication.py

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

qa: add validation stage for deduplication.py

To validate sample-dedup actually works, validate() runs
separated thread from sample-dedup and verifies
two following things.
1. check sample-dedup starts properly.
2. check references of all the chunk objects' in chunk tier
   exists in designated base pool.
This routune repeats for max_valication_cnt times while
sample-dedup is running. If it doesn't raise any fail while the loop,
we can pretend sample-dedup works accurately.
If not, assert() will stop this test.

In case that a reference of chunk object doesn't exist in base pool,
validate() gives a second chance after repairing it (chunk-repair op)
to deal with false-positive reference inconsistency.

Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>

src/tools/ceph_dedup_tool: fixes to daemonize the process correctly

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

qa: add deduplication test under RGW workload (s3)

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #48157 from zdover23/wip-doc-2022-09-18-rados-network-config-prompt-correction

doc/rados: update prompts in network-config-ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: update prompts in network-config-ref

This PR adds unselectable prompts to network-config-
ref.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw: remove unused AioThrottles in rgw_op.cc

throttle creation was moved to RadosStore::get_atomic_writer(),
RadosStore::get_append_writer(), and RadosMultipartUpload::get_writer()

these instances in rgw_op.cc were unused

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa: turn fail_fs off in case default changes

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

win32: remove libcurl

The Windows client does not use libcurl for anything. Remove it to
simplify the build process.

Note, if we ever add libcurl back on Windows, we should disable unused
protocols to harden the build:

  --disable-ftp --disable-ldap --disable-ldaps --disable-rtsp \
  --disable-dict --disable-telnet --disable-tftp --disable-pop3 \
  --disable-imap --disable-smb --disable-smtp --disable-gopher \
  --disable-mqtt --disable-manual --disable-ntlm

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #47849 from torchiaf/fix/37327

mgr/dashboard: Add details to the modal which displays the `safe-to-d…

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #47958 from rhcs-dashboard/edit-host-labels-fix

mgr/dashboard: fix Expected to find element: `cd-modal .badge but never found it

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

win32: skip installing openssl man pages

This speeds up the build a bit and makes the logs easier to read.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #45148 from guits/revert-ceph-crash-changes

Revert "src/ceph-crash.in: remove unused frame in handler()"

Reviewed-by: Teoman ONAY <tonay@redhat.com>

Merge pull request #47965 from cbodley/wip-rgw-drain-cb-capture-error

Revert "rgw: add coroutine drain callback capture_error()"

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

mgr/dashboard: Add details to the modal which displays the `safe-to-destroy` result

- Add warnings type information in the case of the OSDs are not safe to destroy
- Add info type information in the case of the OSDs are safe to destroy

Fixes: https://tracker.ceph.com/issues/37327
Signed-off-by: Francesco Torchia <francesco.torchia@suse.com>

Merge pull request #47878 from xxhdx1985126/wip-pin_set-cpu

crimson/os/seastore/btree: reduce the avg number of queries of pin sets' parent/child lookup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

build: Remove -fno-new-ttp-matchingg flag

This was added in the upgrade to C++17. It's no longer needed since
fixing Clang compatibility got rid of non-conforming templates.

It's no longer needed and getting rid of it is a (minor) quality of
life enhancement since it gets rid of a spurious error when using
Clang based build tools (language server, etc.) while compiling with
GCC.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

qa: mds upgrade suite as per merge-hook scripts

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #47090 from lxbsz/wip-56553

client: do not uninline data for read

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>