doc/dev/crimson: add a section for testing with crimson-store-nbd

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson: add a missing not

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42312 from rzarzynski/wip-crimson-boot_epoch-in-MOSDBoot

crimson/osd: use boot_epoch for MOSDBoot.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42313 from rzarzynski/wip-crimson-drop-temporary-from-pg

crimson/osd: drop a temporary in PG::handle_initialize().

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42303 from tchaikov/wip-crimson-nbd-graceful-shutdown

crimson/tools/store_nbd: fixes to implement graceful shutdown

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #42029 into master

* refs/pull/42029/head:
vstart_runner: use FileNotFoundError when os.stat() fails

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge PR #42030 into master

* refs/pull/42030/head:
vstart_runner: maintain log level when --debug is passed

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #42033 into master

* refs/pull/42033/head:
vstart_runner: add log messages to vstart_runner.py

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/osd: drop a temporary in PG::handle_initialize().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: use boot_epoch for MOSDBoot.

For the sake of compliance and similarity with the classcial OSD.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42267 from ifed01/wip-fix-missing-shared-blob

os/bluestore: fix erroneous SharedBlob record removal during repair.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42209 from sebastian-philipp/doc-dev-cephadm-define-vars

doc/dev/cephadm: Define variables

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42274 from liewegas/cleanup-blkdev

common/blkdev: remove stray debug output

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42206 from ronen-fr/wip-ronenf-list-object

common/hobject: a minor fix and performance gain to hobjects listing

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42210 from sebastian-philipp/options-ms-bind-port-max

common/options: global.yaml: change ms_bind_port_max to 7568

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42268 from neha-ojha/wip-health-cleanup

mon/PGMap: remove get_stuck_counts because there are no callers

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42235 from tchaikov/tools/kvstore-tool

tools/kvstore_tool: add "std::" before ostream and string

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/tools/store_nbd: handle ECONNABORTED returned by accept()

if we abort accept() call, an ECONNABORTED is expected. and we should
handle it, otherwise unhandled exception will be noticed by seastar's
reactor. and it complains in that case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/tools/store_nbd: call segment_manager->close() after tm->close()

TransactionManager::close() calls into journal->close(), which in turn
calls BlockSegmentManager::segment_close(). and
SegmentStateTracker::write_out() is then called by
BlockSegmentManager::segment_close().

but BlockSegmentManager::close() closes the underlying seastar::file,
we are not able to write to the file after closing it.

in this change, to ensure that we can close a segment correctly in
TMDriver::close(), tm->close() is called before
segment_manager->close().

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42269 from zdover23/wip-doc-dev-essentials-irc-2021-07-10

doc/dev: add IRC information to dev guide

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42113 from zdover23/wip-doc-upgrading-ceph-potential-problems-2021-06-30

doc/cephadm: improve "Potential Problems"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42092 from zdover23/wip-doc-upgrading-ceph-starting-the-upgrade-2021-06-29

doc/cephadm: improving "Starting the Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42217 from clwluvw/osd-device-details-grafana

monitoring: fix Physical Device Latency unit

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: mykaul <NOT@FOUND>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #41034 from nSedrickm/auth-storage-directive

mgr/dashboard: create directive for AuthStorage service

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: create directive for AuthStorage service

This commit adds a directive that can be used to conditionally display elements based on authorization/scopes criteria

Fixes: https://tracker.ceph.com/issues/47355
Signed-off-by: Ngwa Sedrick Meh <nsedrick101@gmail.com>

common/blkdev: remove stray debug output

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #41834 from grajoria/master

doc: Correction and improvisation for Timestamp part of the doc

Merge pull request #42272 from smithfarm/wip-51622

rpm: remove macro invocation from comment line

Reviewed-by: Kefu Chai <kchai@redhat.com>

rpm: remove macro invocation from comment line

In RPM spec files, comment lines should not include macro invocations,
because RPM can and will expand them, with unpredictable results.

Fixes: https://tracker.ceph.com/issues/51622
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #42212 from wjwithagen/wjw-fix-signal

rgw: Use signaling compatible with POSIX

doc/dev: add IRC information to dev guide

In days of yore, the Developer Guide linked to the
IRC page at ceph.io. After the 2021 rewriting of
ceph.io, a new era began, and that page was no
longer easily accessible (it could still be found
at old.ceph.com, but this was deprecated).

Anyway, that IRC information should be included in
the docs. This PR makes sure that the IRC
information is included in the docs.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

mon/PGMap: remove get_stuck_counts because there are no callers

The only callers were removed in 729a08923fb4403b88baafa0ae75643cfefe6510,
as a part of the broader health reporting improvements.

Signed-off-by: Neha Ojha <nojha@redhat.com>

os/bluestore: fix erroneous SharedBlob record removal during repair.

Fixes: https://tracker.ceph.com/issues/51619
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #42246 from ceph/1625

doc: 16.2.5 Release Notes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: 16.2.5 Release Notes

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #42253 from liewegas/doc-linode

doc/foundation: add linode

rgw: Use signaling compatible with POSIX

FreeBSD does not have signal_t.
Linux specifies also to use `sigval` as POSIX compliant.

tracker: https://tracker.ceph.com/issues/51610
fixes: https://github.com/ceph/ceph/pull/36266
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #42112 from CongMinYin/fix-common-buffer-SIGABRT

common/buffer: fix stack corruption in rebuild_aligned_size_and_memory()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

common/buffer: fix SIGABRT in  rebuild_aligned_size_and_memory

There is such a bl, which needs to satisfy two conditions:
1)all ptrs' length sum except last ptr is aligned with 4K;
2)the length of last ptr is 0.
This bl will cause stack corruption when calling
bufferlist::rebuild_aligned_size_and_memory().

Deal with this special scenario in rebuild_aligned_size_and_memory() to
solve the bug. And added a specialtest-case to reproduce this scenario.

Fixes: https://tracker.ceph.com/issues/51419
Signed-off-by: Yin Congmin <congmin.yin@intel.com>

Merge pull request #42234 from sseshasa/wip-fix-standalone-cleanups

qa/standalone: Add missing cleanups after completion of a subset of osd and scrub tests.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42034 from neha-ojha/wip-51354

pybind/mgr/progress: fix type of reported_epoch

Reviewed-by: Kamoltat <ksirivad@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41965 from sg7801/master

README.md: fix couple minor syntax issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/foundation: add linode

Signed-off-by: Sage Weil <sage@newdream.net>

README.md: fix couple minor syntax issues

Signed-off-by: Srishti Guleria <guleriasrishti01@gmail.com>

Merge pull request #42055 from leseb/crash-various-fixes

src/ceph-crash.in: various enhancements and fixes

qa/standalone: Add missing teardowns to a subset of osd-scrub-repair tests

Tests identified with missing teardown within osd-scrub-repair.sh:
  1. TEST_periodic_scrub_replicated()
  2. TEST_scrub_warning()
  3. TEST_request_scrub_priority()

Centralize setup and teardown within the run() function for all the tests.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

qa/standalone: Add missing teardowns to a subset of osd tests

The following files and tests in them did not teardown the
cluster after a test completed.

1. osd/osd-force-create.sh
2. osd/osd-reuse-id.sh
3. osd/pg-split-merge.sh

This wouldn't cause issues if the tests are run individually. But when
running all the tests in the files mentioned above, it could introduce
unexpected test failures down the line. For e.g., multiple tests may
create pools with same name and if they are not cleaned up properly, this
could result in unexpected failures in a subsequent test.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

common/hobject: speed up non-printables handling when listing hobjects

Using charconv::to_chars().
Anecdotal speedup, handling an assorted mix of strings: 2.4X
(tested on my laptop)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #42236 from tchaikov/wip-crimson-no-protobuf

ceph.spec,debian: drop protobuf dependencies

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42185 from qiuxinyidian/dev2

cmake: set denc_plugin_dir with the full path

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.spec,debian: drop protobuf dependencies

since Seastar has dropped the protobuf dependencies, there is no
need to prepare them for building crimson anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: bump seastar to latest upstream master

to pick up the change to drop protobuf dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/kvstore_tool: add "std::" before ostream and string

also use <iosfwd> instead of <iostream> for forward declaration of
std::ostream, so that we don't need to include the whole iostream
header file in kvstore_tool.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42141 from trociny/wip-38931

osd: move down peers out from peer_purged

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #42164 from yuriw/wip-yuriw-crontab-master

qa/tests: removed nautilus suites as EOL

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

qa/tests: removed nautilus suites as EOL, kept kcephfs and krbd suites

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #42176 from tchaikov/wip-doc-option

doc/rados/troubleshooting/log-and-debug: document option using :confval:

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42136 from dvanders/dvanders_lec_merging

mon/OSDMonitor: resize oversized Lec::epoch_by_pg, after PG merging, preventing osdmap trimming

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

monitoring: fix Physical Device Latency unit

Based on the expr it should be seconds

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #42190 from ivancich/wip-unused-store-warning

rgw: remove unused store variable

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kalpesh Pandya <kapandya@redhat.com>

mon/OSDMonitor: account for PG merging in epoch_by_pg accounting

After a pool has merged PGs, the epoch_by_pg accounting will refer
to osdmap epochs of PGs that no longer exist. We'll never again get
OSD beacons for these PGs, so the min epoch in epoch_by_pg will not
advance until the mon leader has restarted. The effect of this is
that osdmaps are not trimmed after a pool has undergone PG merging,
until the mon leader restarts. To fix, we unconditionally resize
epoch_by_pg to the pg_num of the pool during each beacon report.

Fixes: https://tracker.ceph.com/issues/48212
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #42040 from wjwithagen/wjw-wip-bluestore-choose_fdv3

blk: use choose_fd for all filehandle references

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/options: global.yaml: change ms_bind_port_max to 7568

If the cluster is set to have very dense nodes (>60 OSDs per host)
we have to make sure to assign sufficient ports for Ceph OSDs. The
current default (6800-7300) currently allows for no more than 62 OSDs
per host. For cluster with dense nodes like this we have to adjust the
setting "ms_bind_port_max" to a suitable value. Each OSD will consume
8 additional ports. For example, given a host that is set to run 96 OSDs,
768 ports will be needed. "ms_bind_port_max" should be set at least to
7568.

Fixes: https://tracker.ceph.com/issues/48292
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

blk/kernel: Only use file hint capabilities if available.

Without WRITE_LIFE capabilities, only one file is used.
And rocksdb sets this value also to > 0, so we need to catch this here
instead of trusting rocksdb to set write_hint.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

blk/kernel: reorganise and use fd in debug

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

blk: use choose_fd for all filehandle references

This is a leading part of the changes to implement bluestore on
FreeBSD. Without this invalid indexing in the descriptor arrays
will occur.

Creates a new enum: `blk_access_mode_t`
to describe `BUFFERED` and `DIRECT `mode access
with `choose_fd()` to get the correct file for the typed access
and adds
a pretty-printer
boolean convertor `blk_access_mode_t::buffermode(bool)`

This PR is a redo for PR #37258, since that one was lost
in rebasing errors. But the review notes are still there.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc/dev/cephadm: Define variables

Fixes: https://tracker.ceph.com/issues/47142
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42183 from SMIL-Infra/fix-ac-lang

mgr/dashboard: fix Accept-Language header parsing

Merge pull request #36266 from amathuria/wip-rgw-d3n-cache-upstreaming

rgw: D3N Cache changes for Upstream

Merge pull request #42158 from tchaikov/wip-e2e-centos

qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #42175 from tchaikov/wip-45457

log: set hostname and fsid for graylog

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42205 from tchaikov/wip-doc-crimson-metrics

doc/dev/crimson: add a section for metrics

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #42179 from brucen1030/wip-fix-mon-crash-when-osd-pool-get

mon: return -EINVAL when handling unknown option in 'ceph osd pool get'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42177 from sebastian-philipp/cephadm-ls-garbage

mgr/cephadm: avoid saving daemons of unknown type

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev/crimson: add a section for metrics

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: set denc_plugin_dir with the full path

This commit changes denc_plugin_dir from relative to absolute path, so
that CEPH_DENC_MOD_DIR points to the absolute path and ceph-dencoder
is able to load the shared libraries from the specified path.

Fixes:https://tracker.ceph.com/issues/51519
Signed-off-by: zhipeng li <qiuxinyidian@gmail.com>

Merge pull request #42168 from tchaikov/wip-mgr-unsigned-pgp

mgr/DaemonServer: use "unsigned" to represent the pgp num

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #42203 from tchaikov/wip-crimson-metrics

crimson/admin: dump DERIVE metrics as signed integer

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/admin: dump DERIVE metrics as signed integer

per include/seastar/core/metrics.hh,

// The value binding data types
enum class data_type : uint8_t {
    COUNTER, // unsigned int 64
    GAUGE, // double
    DERIVE, // signed int 64
    ABSOLUTE, // unsigned int 64
    HISTOGRAM,
};

the DERIVE values are signed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #42139 into master

* refs/pull/42139/head:
src/pybind/mgr/mirroring/fs/snapshot_mirror.py: do not assume a cephfs-mirror daemon is always running

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #42137 into master

* refs/pull/42137/head:
msg/async/net_handler: do not define variable unless it is used
mds/cephfs_features: print size_t using "%zu"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42155 from yuvalif/wip-yuval-fix-51347

rgw/notifications: cache object size to avoid accessing invalid memoery

Merge PR #41966 into master

* refs/pull/41966/head:
mgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: D3N Cache changes for Upstream

Upstreaming / rebase of #24500

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: change io_ctx pool per storage class

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Changing free() to delete()

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Addressing review comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Fixing seg fault

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Moving CacheRequest out of librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cache initialization fix

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix missing spawn.hpp compile errors

resolves compilation errors similar to:
```
[ 15%] Building CXX object src/tools/immutable_object_cache/CMakeFiles/ceph_immutable_object_cache_lib.dir/CacheController.cc.o
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_common.h:31,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_rados.h:17,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/IoCtxImpl.h:30,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/RadosClient.h:35,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOSImpl.h:27,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOS.cc:37:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
   31 | #include <spawn/spawn.hpp>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
src/neorados/CMakeFiles/neorados_api_obj.dir/build.make:62: recipe for target 'src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o' failed
make[3]: *** [src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o] Error 1
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Resolving merge conflict

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Removing all logs and unnecessary comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Cache Read and Write working

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Initial Commit L1 Cache

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
post re-rebase merge, update fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixup! post re-rebase merge, update fixes

rgw: continuation of rgwcache branch rebase

Signed-off-by: Mark Kogan <mkogan@redhat.com>
RGW: DataCache: post merge fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixes of segmentation fault caused by oid

Signed-off-by: E. Ugur Kaynar <ukaynar@bu.edu>
rgw: fixes for segmentation faults and configuration processing

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: data cache first commit

Signed-off-by: Mania Abdi <mania.abdi287@gmail.com>
rgw: cleanup addressing PR comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: cleanup addressing PR comments, continuation.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: pr cleanup addressing second review round

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Addressing review comments, removing all D3N code from librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: for compilation err from removal of mydout() helper

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rge: addressing review comments

rgw: move d3n datacache into separate files
rgw: 1st part of datacache rebranding to d3n

fix forward declaration compile err (only with clang):
../src/rgw/rgw_cache.h:396:4: error: member access into incomplete type 'struct get_obj_data'
  d->data_lock.lock();
   ^
../src/rgw/rgw_cache.h:365:8: note: forward declaration of 'get_obj_data'
struct get_obj_data;
       ^

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: addressing review comments, datacache rebranding to d3n cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Cleaning up unused D3N cache code

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cont. cleaning up of rgw_obj_data()

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Removing redundant code, fix for multipart S3 objects

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix: incorrect content len on multipart get

in s3tests_boto3.functional.test_s3:test_multipart_copy_versioned
when d3n cache is disabled

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault reading from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite

sync on secondary site

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite teuthology tests, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Adding drain to wait for all AIO reads to complete

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix for using read() by liabio or posix io per config

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: improve persistent data cache directory handling

on start create the persistent datacache directory if necessary
and add an option to evict it's content if already exists

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix possible segfault during eviction

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add debug logs for cache in/out flow

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: move the L2 cache functionality to separate PR

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address java_s3tests teuthology issues

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: do not handle compressed objects fro now

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: l2 cleanup and log fixups + post dpp

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: thread dpp thru get_obj_iterate_cb() and related

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: bypass reading versioned objects from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: cleanup and fix s3tests

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: allow to enable cache only on beast

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: delete the content of the cache directory on rgw start

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: re-enable d3n cache with civetweb frontend

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: rebase post zipper 10

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address teuthoogy valgrind leaks detected

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigating valgrind leaks

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: remove rgw_d3n_l1_libaio_read option

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip segfault fix

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigate libaio SIGEV_THREAD cb race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: change libaio signaling mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip cont. libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio cleanups and edge case handling fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: narrow the libaio locking scope

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add libaio req ordering mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix lock regression

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing reviwe comments and cleasnup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio locks cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: refactor libaio abstraction to share the ioc implementation

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments and cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address review comments, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Ali Maredia <amaredia@redhat.com>
Co-authored-by: Feng Hualong <hualong.feng@intel.com>

common/hobject: fix the handling of non-printable chars when listing hobjects

The 'char >= 127' test was always(*) false.

(*) ostensibly - compiler dependent

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #42174 from cyx1231st/wip-seastore-fix-transactional-retire

crimson/os/seastore: leverage RetiredExtentPlaceholder to detect transaction conflicts

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cache: cleanup query_cache_with_placeholders()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction: remove unused retired_uncached

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: leverage RetiredExtentPlaceholder to detect transaction conflicts

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: remove unused template in RetiredExtentPlaceholder

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: cleanup, consolidate extent lookup through query_cache_for_extent()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #42187 from tchaikov/wip-crimson-cleanup

crimson/os/seastore: use discard_result() to discard futurized value

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: remove unused store variable

A recent change made the store variable unnecessary and now compiling
generates a warning. Clean up.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

doc/rados/troubleshooting/log-and-debug: use ini to highlight code block

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/troubleshooting/log-and-debug: document option using :confval:

* simpler this way.
* and drop non-existent options.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42186 from lorenzbausch/fix_typo

doc/dev/crimson/osd: fix typo

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/common/errorator: forward declare futurize as a struct

as it is defined as struct, otherwise clang complains on this
inconsistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: use discard_result() to discard futurized value

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: capture variables explicitly

less error prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: add discard_result() to errorator

so we can discard result explicitly.

Signed-off-by: Kefu Chai <kchai@redhat.com>