crimson/os: use fmt::format() instead of sstream

for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #45573 from wxypro/cephfs-shell-typos

tools/cephfs-shell: fix typos in comments

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44895 from rkachach/fix_issue_54026

mgr/cephadm: Adding support for natural sorting

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #45241 from rkachach/fix_issue_51072

mgr/cephadm: fixing prometheus port handling

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45314 from rkachach/fix_issue_54503

mgr/cephadm: checking service name before removal

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45601 from idryomov/wip-diff-iterate-striping-fix

librbd: make diff-iterate in fast-diff mode sort and merge reported extents

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #45513 from tchaikov/wip-cxx20-fixes-mon

mgr,mon,test: C++20 fixes

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45386 from cyx1231st/wip-crimson-epm-split-hot-cold

crimson/os/seastore: reuse RecordSubmitter for SegmentedJournal and EPM::Writer

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #45115 from mchangir/mgr-snap_schedule-restart-old-schedules-on-startup

mgr/snap_schedule: restart old schedules

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45399 from adk3798/no-firewall

cephadm: respect --skip-firewalld flag

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #45401 from adk3798/legacy-config

cephadm: verify config file exists when inferring it

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #45527 from adk3798/eca-osd-allowed

python-common/drive_group: add extra_container_args to supported features

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45250 from ifed01/wip-ifed-get-bluest-extents-cleanup

os/bluestore: refactor bluefs get_block_extents interface

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #45393 from rzarzynski/wip-crimson-fix-throw-in-plugins

build, crimson/osd: do not let Seastar to interfere with ELF's program headers

Reviewed-by: Samuel Just <sjust@redhat.com>

qa: test snap_schedule with mgr restart

Scheduled snaps should follow the created schedule even across mgr
restart.

Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/snap_schedule: restart old schedules

Old schedules were not picked up from database when mgr was restarted.
Restart old schedules on mgr restart.

Fixes: https://tracker.ceph.com/issues/54052
Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/util: add function to list all fs names

Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/cephadm: fixing natsort dependencies on debian

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: Adding support for natural sorting
Fixes: https://tracker.ceph.com/issues/54026
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45472 from vshankar/revert-45307

Revert "mds: kill session when mds do ms_handle_remote_reset"

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Revert "mds: kill session when mds do ms_handle_remote_reset"

This reverts commit 3651deb4e0b0c102adcaddce79ee4e053f033418.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45340 from benhanokh/disable_ncb_rotational

os/bluestore: Disable NCB functionality on rotational drives

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

build, crimson/osd: do not let Seastar to interfere with ELF's program headers.

For the sake of avoiding locking on the `__cxa_throw` paths, Seastar
hijacks `dl_iterate_phdr` of the dynamic linker. Unfortunately, this
has a nasty side effect: it makes impossible to catch an exception in
in a plugin (a DSO loaded via the `dlopen()` machinery).

For mote details please consult:
  * https://gist.github.com/rzarzynski/3abe9ed6b50cfa1893d34988e1628bfc,
  * `seastar/src/core/exception_hacks.cc`.

This patch deals with the problem by simply disabling the problematic
workaround which could be iatrogenic too. If that would be the case,
we can consider:

  * preloading all our Ceph Classes before reaching `smp::configure()`,
  * statically linking them.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #45551 from liu-chunmei/crimson-string-view

crimson: keep string alive during string_view lifetime

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44696 from cbodley/wip-arrow-submodule-ext

cmake: add submodule for Apache Arrow at v6.0.1

Reviewed-by: Gal Salomon <gsalomon@redhat.com>

Merge pull request #45550 from liu-chunmei/crimson-fix-read-error-return

crimson: return read error code in MOSDOpReply

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #45246 from aclamk/wip-bluefs-sync-compaction-fix

Fix BlueFS sync compaction

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #45230 from ljflores/wip-github-actions-stale

.github/workflows: replace probot-stale with equivalent github action

Merge pull request #45466 from neha-ojha/wip-54592

osd/PrimaryLogPG.cc: CEPH_OSD_OP_OMAPRMKEYRANGE should mark omap dirty

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #45366 from NitzanMordhai/wip-nitzan-test-librados-watchnotify-watch2delete

tests: ceph_test_rados_api_watch_notify: watch2Delete reconnect

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

librbd: make diff-iterate in fast-diff mode sort and merge reported extents

Various users, the most notable example being the QEMU driver, assume
that extents are reported in image offset order.

Fixes: https://tracker.ceph.com/issues/53885
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45468 from idryomov/wip-pool-reverse-lookup-osdmap

librados: check latest osdmap on ENOENT in pool_reverse_lookup()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #45429 from votdev/issue_54591_language_dropdown

mgr/dashboard: Language dropdown box is partly hidden on login page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #45577 from wxypro/mgr-stat-fix

mgr/stats: fix a typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

mgr/cephadm: fixing prometheus port handling
Fixes: https://tracker.ceph.com/issues/51072
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #44579 from asm0deuz/fix_pids_limits_bug_52898

cephadm: Remove containers pids-limit

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/stats: fix an typo in comment

fix an typo in comment

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

crimson: keep string alive during string_view lifetime

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

crimson:return read error code in MOSDOpReply

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

tools/cephfs-shell: fix typos in comments

fix typos in comments

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

Merge pull request #45512 from zdover23/wip-doc-2022-03-18-master-to-main

ceph/admin: s/master/main

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephadm: verify config file exists when inferring it

Fixes: https://tracker.ceph.com/issues/54571
Signed-off-by: Adam King <adking@redhat.com>

cephadm: respect --skip-firewalld flag

Fixes: https://tracker.ceph.com/issues/54137
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #45286 from adk3798/ssh-offline

mgr/cephadm: add keep-alive requests to ssh connections

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45333 from s0nea/config-fqdn

mgr/cephadm: try to get FQDN for configuration files

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

cephadm: remove containers pids-limit

The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>

Merge pull request #45269 from kalaspuffar/table_fix

mgr/dashboard: Table columns hiding fix

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: kalaspuffar <NOT@FOUND>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #45510 from ceph/fix-github_workflow-master

.github: continue on error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44616 from dang/wip-dang-zipper-objctx

RGW - Zipper - remove RGWObjectCtx from SAL API

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

test: Remove unused variable

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Pragma to silence infinite recursion warning for GCC

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Initialize possibly uninitialized buffer

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #45533 from rhcs-dashboard/update-grafana-version

mgr/dashboard/monitoring: update grafana version

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

mgr: use constexpr string_view as input to format()

resolves a c++20 compilation error with clang13:

ceph/src/mgr/Mgr.cc:602:42: error: call to consteval function 'fmt::basic_format_string<char, const std::basic_string<char> &>::basic_format_string<const char *, 0>' is not a constant expression
    monc->start_mon_command({fmt::format(cmd, name)}, {},
                                         ^
ceph/src/fmt/include/fmt/core.h:2839:56: note: read of non-constexpr variable 'cmd' is not allowed in a constant expression
  FMT_CONSTEVAL basic_format_string(const S& s) : str_(s) {
                                                       ^
ceph/src/mgr/Mgr.cc:602:42: note: in call to 'basic_format_string(cmd)'
    monc->start_mon_command({fmt::format(cmd, name)}, {},
                                         ^
ceph/src/mgr/Mgr.cc:601:17: note: declared here
    const char* cmd = R"({{"prefix": "mon metadata", "id": "{}"}})";
                ^

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mon: Replace deprecated use of format_to

The non-deprecated version takes an explicit OutputIterator.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Fix ranged-construct warning

In the case of an uint32_t there's no benefit to a reference so just
remove the const and make the copy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Replace deprecated spirit/include/phoneix.hpp with phoenix.hpp

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mgr/cephadm: checking service name before removal
Fixes: https://tracker.ceph.com/issues/54503
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45420 from mgfritch/cephadm-infer-image-pull

cephadm: infer the default container image during pull

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #45506 from dang/wip-dang-53614

RGW - Make admin bucket chown accept marker

reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45360 from rhcs-dashboard/fix-54481-master

mgr/dashboard: fix wrong pg status processing

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #45353 from PepperJo/readv_writev_overflow_v2

librbd: readv/writev fix iovecs length computation overflow

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45285 from chrisphoffman/wip-rbd-du-validate

test/librbd: add test to verify diff_iterate size

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45221 from rzarzynski/wip-core-drop-unused-aiocompletion-in-librados

core, test: fix compile-time warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45415 from Matan-B/wip-matanb-watch-r-buffer

crimson/osd: fix osd crash due to buffer reference binding

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #45050 from hualongfeng/qatchange

common/compressor: fix the issue that cannot processed concurrently

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44491 from nmshelke/fix-46075

ceph-fuse: ignore fuse mount failure if path is already mounted

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Fix a problem in store_test::BluestoreBrokenNoSharedBlobRepairTest where the check for active null-fm was wrong and so reporting bogus errors  when null-fm was inactive
The check need to access dynamic value and not config setting (which can be overridden)
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>

mgr/dashboard/monitoring: update grafana version

Fixes: https://tracker.ceph.com/issues/54311
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #45406 from wxypro/fix-typos

docs,options: fix typos

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45525 from idryomov/wip-rbd-trash-purge-schedule-relax-assert

qa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #45517 from liu-chunmei/crimson-fix-cmpxattr

crimson: let read operation return result in MOSDOpReply

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

python-common/drive_group: add extra_container_args to supported features

Should have been added when extending extra container args
to all the services but was missed

Signed-off-by: Adam King <adking@redhat.com>

common/compressor: fix the issue that cannot processed concurrently

Now, one session cannot support concurrent and it will lead to crash.
So there are mutil session using. At same time, it also can improve
the performance.

Fixes: https://tracker.ceph.com/issues/54361
Signed-off-by: Feng Hualong <hualong.feng@intel.com>

qa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert

Commit 08df6e0fd006 ("qa/workunits/rbd: expand LevelSpec parsing
coverage") didn't account for images with a separate data pool.  This
was missed because of small-cache-pool.yaml breakage.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

crimson: let read operation return result in MOSDOpReply

otherwise cmp_xattr will failed,
only check if return result for write operation.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #45518 from liu-chunmei/crimson-fix-read-length

crimson: set returned read length in op.extent.length not in rval

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #45520 from rzarzynski/wip-crimson-rvalue-in-future-set_callback

crimson/common, seastar: bump up to a newer version. Fix a FTBFS

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44835 from rhcs-dashboard/cypress-cucumber

mgr/dashboard: BDD approach for the dashboard cephadm e2e

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #45345 from cbodley/wip-cls-rgw-suggest-version

cls/rgw: rgw_dir_suggest_changes detects race with completion

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #45300 from soumyakoduri/wip-skoduri-lc-sync

rgw: Update "CEPH_RGW_DIR_SUGGEST_LOG_OP" for remove entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

mgr/cephadm: try to get FQDN for inventory address

Fixes: https://tracker.ceph.com/issues/54502
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/cephadm: unify way to get the host address

There are two different ways to get the host address. From the
inventory of the mgr object directly or via the `_inventory_get_addr`
method of `CephadmService`. Update the code in order to use the
`_inventory_get_addr` method only.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

seastar: bump up to a newer version.

It's needed to addresses a FTBFS due to the Seastar's
no-locking-when-throwing hack.

Tags: seastar submodule
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/common: pass rvalue reference to future::set_callback().

It's necessary since 710a1bfdc02202fe9e59df8ea31de5b82b893fb4
in Seastar.

This change is a part of ongoing upgrade of Seastar which will
be completed in a follow-up PR, after merging another change
with the Seastar's upstream.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

librbd: readv/writev fix iovecs length computation overflow

iovec have unsigned length (size_t) and before this patch the
total length was computed by adding iovec's length to a signed
length variable (ssize_t). While the code checked if the resulting
length was negative on overflow, the case where length is positive
after overflow was not checked. This patch fixes the overflow check
by changing length to unsigned size_t.

Additionally, this patch fixes the case where some iovecs have been
added to the bufferlist and the aio completion has been blocked, but
adding an additional iovec fails because of overflow. This leads to
the UserBufferDeleter trying to unblock the completion on destruction
of the bufferlist but asserting because the completion was never
armed. We avoid this by first computing the total length and checking
for overflows and iovcnt before adding them to the bufferlist.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>

tests: ceph_test_rados_api_watch_notify: watch2Delete reconnect

During test LibRadosWatchNotify.Watch2Delete rados_watch_check can return error -102 if reconnect happened, in that case Broken pipe reconnect and -102 returned

Fixes: https://tracker.ceph.com/issues/51307
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
Signed-off-by: NitzanMordhai <nmordech@redhat.com>

test/crimson/seastore: fix metrics registration conflicts

The conflicts are from adding segment manager to transaction manager
repeatedly. Fixed the repeat adding issue, add ceph_assert() to detect
this issue as early as possible, and consolidate initiation code into
make_transaction_manager().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/journal/segment_allocator: fix metrics for RecordSubmitter

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/journal/segment_allocator: introduce open() and close() to RecordSubmitter

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/EPM: clear allocators when close

SeaStore::mkfs() and mount() will both invoke
TransactionManager::add_segment_manager(). Without clearing allocators,
EPM will add the same allocator multiple times.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/../segment_allocator: improve log name with device-id, cleanup

Also clean up segment type related special handling.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: assign the missing allocation hint

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/EPM: coalesce writes using the same writer

After integrated with RecordSubmitter, Writer is able to coalesce
concurrent writes into record groups. There is no need to distribute
writes to multiple Writers for both large and small write sizes.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/EPM: integrate Writer with RecordSubmitter

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/EPM: introduce open() and close()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson: set returned read length in op.extent.length not in rval

rval set to be zero, means read success.
otherwise CReadOpsTest failed.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

crimson/os/seastore/EPM: cleanup, drop ool_record_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #45419 from tchaikov/wip-cxx20-fixes-core

osd, common: C++20 related fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>