doc/dpdk: reword the root access part

the root access to system is not a must have for running a DPDK
application. so reword the "Configuring OSD DPDKStack" section.

also, manually editing /etc/passwd is not encouraged, so use
"usermod" instead. to add a directory after user's command
interpreter in /etc/passwd does not make sense. see PASSWD(5).
so drop the paragraph on editing /etc/passwd.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

doc/dpdk: improve the formatting

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44292 from fengchunsong/dpdk-doc

doc/dev: add dpdkstack doc

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44343 from adk3798/nfs-offline

mgr/cephadm: Reschedule nfs daemons from offline hosts

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #45649 from tchaikov/wip-crimson-cleanup

crimson/os: use fmt::format() instead of sstream

Merge pull request #45539 from rhcs-dashboard/css-fix-highlight-search

mgr/dashboard: Remove padding in search highlighted text

Merge pull request #45651 from sseshasa/wip-qa-fix-activate-osd-test

qa/standalone: Fix test_activate_osd() test in ceph-helpers.sh

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #45553 from wxypro/doc-update

doc/mgr/prometheus.rst: fix spelling error

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #45575 from wxypro/options-fix

common/options: fix typos: smarctl -> smartctl

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44699 from ivancich/wip-bucket-index-timeout-config

core,rgw: allow configuration of bi tx timeout; add instrumentation

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #45522 from sseshasa/wip-improve-mclock-docs

Doc: Improve mclock config reference documentation & update PendingReleaseNotes.

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/standalone: Fix test_activate_osd() test in ceph-helpers.sh

Modify test_activate_osd() to get the type of scheduler in use and then
verify the value of osd_max_backfills. This is because mclock scheduler
overrides this option to 1000 upon OSD initialization.

The test earlier used to pass because the OSD daemon was killed but not
marked down and upon being brought up, the wait for OSD up check was
passing quickly. But the OSD still didn't have the latest config values.

But now upon killing the OSD, the osd_fast_shutdown sequence notifies the
mon (see PR: https://github.com/ceph/ceph/pull/44807) and is marked down
and dead. Upon bringing it up, the wait for OSD up check takes a longer
time and this is sufficient for the config values to be updated. This
results in the correct values being read from the config 'Values' map.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #44807 from NitzanMordhai/wip-nitzan-fast-shutdown-notify-mon

osd/OSD: osd_fast_shutdown_notify_mon not quite right

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

crimson/os: use fmt::format() instead of sstream

for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #45309 from caisan/modify_without_source_segfault

radosgw-admin: fix segfault on pipe modify without source/dest zone specified

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45573 from wxypro/cephfs-shell-typos

tools/cephfs-shell: fix typos in comments

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44895 from rkachach/fix_issue_54026

mgr/cephadm: Adding support for natural sorting

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #45241 from rkachach/fix_issue_51072

mgr/cephadm: fixing prometheus port handling

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45314 from rkachach/fix_issue_54503

mgr/cephadm: checking service name before removal

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45601 from idryomov/wip-diff-iterate-striping-fix

librbd: make diff-iterate in fast-diff mode sort and merge reported extents

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #45513 from tchaikov/wip-cxx20-fixes-mon

mgr,mon,test: C++20 fixes

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45386 from cyx1231st/wip-crimson-epm-split-hot-cold

crimson/os/seastore: reuse RecordSubmitter for SegmentedJournal and EPM::Writer

Reviewed-by: Samuel Just <sjust@redhat.com>

rgw: bucket index transaction instrumentation clean-ups

Add a series of clean-ups to the instrumentation code. Keep this as a
separate commit so it will be easier to reconcile other code streams.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: additional configurable CLS instrumentation on bucket index TXs

This supplements an earlier commit to add additional instrumentation
on bucket index transactions on the CLS side.

The instrumentation is triggered by setting the configuration option
rgw_bucket_index_transaction_instrumentation in the [global] section.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #45115 from mchangir/mgr-snap_schedule-restart-old-schedules-on-startup

mgr/snap_schedule: restart old schedules

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45399 from adk3798/no-firewall

cephadm: respect --skip-firewalld flag

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #45401 from adk3798/legacy-config

cephadm: verify config file exists when inferring it

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #45527 from adk3798/eca-osd-allowed

python-common/drive_group: add extra_container_args to supported features

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45250 from ifed01/wip-ifed-get-bluest-extents-cleanup

os/bluestore: refactor bluefs get_block_extents interface

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

rgw: configurable instrumentation on bucket index transactions

In order to better understand corner cases with bucket index
operations, extra instrumentation is now added and controlled by a
boolean configuration variable
("rgw_bucket_index_transaction_instrumentation").

When set to true, there is extra logging during all CLS operations
involving bucket index transactions. Additionally, all these log
entries are tagged with "BITX" to make them easier to find in the
logs. This is preferable to setting all OSD logging at a high level
due to the log size issues.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: make bucket index pending op expiration configureable

Bucket index operations are transactional with data object
manipulation. The operation is prepared by adding a pending operation
record. And when the data object side is complete, the bucket index
operation is committed.

If it fails to be committed, later bucket listings will compare the
pending ops with the current data object state and see whether it
completed or not and then either commit or expire the op. The time
span for expiration is currently hard-coded as 120 seconds (unless
overridden in the bucket header, which can happen during "bucket
check").

This commit allows that expiration time to be configured.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

osd: add new CLS call to retrieve global configuration

Currently there is no easy way to gain access to global configuration
from CLS (objclass) code. This adds a new call to the CLS interface
that returns a "const ConfigProxy&" from which configuration can be
accessed.

NOTE: Working code to do provide this functionality in crimson is not
provided.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #45393 from rzarzynski/wip-crimson-fix-throw-in-plugins

build, crimson/osd: do not let Seastar to interfere with ELF's program headers

Reviewed-by: Samuel Just <sjust@redhat.com>

qa: test snap_schedule with mgr restart

Scheduled snaps should follow the created schedule even across mgr
restart.

Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/snap_schedule: restart old schedules

Old schedules were not picked up from database when mgr was restarted.
Restart old schedules on mgr restart.

Fixes: https://tracker.ceph.com/issues/54052
Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/util: add function to list all fs names

Signed-off-by: Milind Changire <mchangir@redhat.com>

PendingReleaseNotes: Add mclock config reference link to an existing note

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

doc: Improvements to mClock configuration reference documentation

Improve the documentation around.
 - mclock client types.
 - Describe in greater detail about mclock config profiles.
 - Add notes about manually benchmarking OSDs and tuning bluestore throttle
   parameters.
 - Include a couple of missing mclock configuration options.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

mgr/cephadm: fixing natsort dependencies on debian

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/cephadm: Adding support for natural sorting
Fixes: https://tracker.ceph.com/issues/54026
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45472 from vshankar/revert-45307

Revert "mds: kill session when mds do ms_handle_remote_reset"

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Revert "mds: kill session when mds do ms_handle_remote_reset"

This reverts commit 3651deb4e0b0c102adcaddce79ee4e053f033418.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45340 from benhanokh/disable_ncb_rotational

os/bluestore: Disable NCB functionality on rotational drives

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

build, crimson/osd: do not let Seastar to interfere with ELF's program headers.

For the sake of avoiding locking on the `__cxa_throw` paths, Seastar
hijacks `dl_iterate_phdr` of the dynamic linker. Unfortunately, this
has a nasty side effect: it makes impossible to catch an exception in
in a plugin (a DSO loaded via the `dlopen()` machinery).

For mote details please consult:
  * https://gist.github.com/rzarzynski/3abe9ed6b50cfa1893d34988e1628bfc,
  * `seastar/src/core/exception_hacks.cc`.

This patch deals with the problem by simply disabling the problematic
workaround which could be iatrogenic too. If that would be the case,
we can consider:

  * preloading all our Ceph Classes before reaching `smp::configure()`,
  * statically linking them.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #45551 from liu-chunmei/crimson-string-view

crimson: keep string alive during string_view lifetime

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44696 from cbodley/wip-arrow-submodule-ext

cmake: add submodule for Apache Arrow at v6.0.1

Reviewed-by: Gal Salomon <gsalomon@redhat.com>

Merge pull request #45550 from liu-chunmei/crimson-fix-read-error-return

crimson: return read error code in MOSDOpReply

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #45246 from aclamk/wip-bluefs-sync-compaction-fix

Fix BlueFS sync compaction

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #45230 from ljflores/wip-github-actions-stale

.github/workflows: replace probot-stale with equivalent github action

Merge pull request #45466 from neha-ojha/wip-54592

osd/PrimaryLogPG.cc: CEPH_OSD_OP_OMAPRMKEYRANGE should mark omap dirty

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #45366 from NitzanMordhai/wip-nitzan-test-librados-watchnotify-watch2delete

tests: ceph_test_rados_api_watch_notify: watch2Delete reconnect

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

librbd: make diff-iterate in fast-diff mode sort and merge reported extents

Various users, the most notable example being the QEMU driver, assume
that extents are reported in image offset order.

Fixes: https://tracker.ceph.com/issues/53885
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45468 from idryomov/wip-pool-reverse-lookup-osdmap

librados: check latest osdmap on ENOENT in pool_reverse_lookup()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/OSD: osd_fast_shutdown_notify_mon not quite right

When osd_fast_shutdown and osd_fast_shutdown_notify_mon set as true, OSD marked as Down
it should be marked as Dead,

Fixed: https://tracker.ceph.com/issues/53327

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
nd

nd

mgr/dashboard: Remove padding in search highlighted text

Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>

Merge pull request #45429 from votdev/issue_54591_language_dropdown

mgr/dashboard: Language dropdown box is partly hidden on login page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #45577 from wxypro/mgr-stat-fix

mgr/stats: fix a typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

mgr/cephadm: fixing prometheus port handling
Fixes: https://tracker.ceph.com/issues/51072
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #44579 from asm0deuz/fix_pids_limits_bug_52898

cephadm: Remove containers pids-limit

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/stats: fix an typo in comment

fix an typo in comment

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

common/options: fix typos: smarctl -> smartctl

fix typos: smarctl -> smartctl

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

crimson: keep string alive during string_view lifetime

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

crimson:return read error code in MOSDOpReply

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

tools/cephfs-shell: fix typos in comments

fix typos in comments

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

Merge pull request #45512 from zdover23/wip-doc-2022-03-18-master-to-main

ceph/admin: s/master/main

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephadm: verify config file exists when inferring it

Fixes: https://tracker.ceph.com/issues/54571
Signed-off-by: Adam King <adking@redhat.com>

cephadm: respect --skip-firewalld flag

Fixes: https://tracker.ceph.com/issues/54137
Signed-off-by: Adam King <adking@redhat.com>

mgr/cephadm: Reschedule nfs daemons from offline hosts

In order to improve nfs availability, if there are other
hosts we can place an nfs daemon on or if there is a host
with a lower rank nfs daemon when a higher rank one is on
an offline host, we should reschedule the nfs daemons

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #45286 from adk3798/ssh-offline

mgr/cephadm: add keep-alive requests to ssh connections

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45333 from s0nea/config-fqdn

mgr/cephadm: try to get FQDN for configuration files

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

cephadm: remove containers pids-limit

The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>

Merge pull request #45269 from kalaspuffar/table_fix

mgr/dashboard: Table columns hiding fix

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: kalaspuffar <NOT@FOUND>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #45510 from ceph/fix-github_workflow-master

.github: continue on error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #44616 from dang/wip-dang-zipper-objctx

RGW - Zipper - remove RGWObjectCtx from SAL API

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

test: Remove unused variable

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Pragma to silence infinite recursion warning for GCC

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test: Initialize possibly uninitialized buffer

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #45533 from rhcs-dashboard/update-grafana-version

mgr/dashboard/monitoring: update grafana version

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

mgr: use constexpr string_view as input to format()

resolves a c++20 compilation error with clang13:

ceph/src/mgr/Mgr.cc:602:42: error: call to consteval function 'fmt::basic_format_string<char, const std::basic_string<char> &>::basic_format_string<const char *, 0>' is not a constant expression
    monc->start_mon_command({fmt::format(cmd, name)}, {},
                                         ^
ceph/src/fmt/include/fmt/core.h:2839:56: note: read of non-constexpr variable 'cmd' is not allowed in a constant expression
  FMT_CONSTEVAL basic_format_string(const S& s) : str_(s) {
                                                       ^
ceph/src/mgr/Mgr.cc:602:42: note: in call to 'basic_format_string(cmd)'
    monc->start_mon_command({fmt::format(cmd, name)}, {},
                                         ^
ceph/src/mgr/Mgr.cc:601:17: note: declared here
    const char* cmd = R"({{"prefix": "mon metadata", "id": "{}"}})";
                ^

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mon: Replace deprecated use of format_to

The non-deprecated version takes an explicit OutputIterator.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Fix ranged-construct warning

In the case of an uint32_t there's no benefit to a reference so just
remove the const and make the copy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Replace deprecated spirit/include/phoneix.hpp with phoenix.hpp

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mgr/cephadm: checking service name before removal
Fixes: https://tracker.ceph.com/issues/54503
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #45420 from mgfritch/cephadm-infer-image-pull

cephadm: infer the default container image during pull

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #45506 from dang/wip-dang-53614

RGW - Make admin bucket chown accept marker

reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45360 from rhcs-dashboard/fix-54481-master

mgr/dashboard: fix wrong pg status processing

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

doc/mgr/prometheus.rst: fix spelling error

fix spelling error

Signed-off-by: wangxinyu <wangxinyu@inspur.com>

Merge pull request #45353 from PepperJo/readv_writev_overflow_v2

librbd: readv/writev fix iovecs length computation overflow

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45285 from chrisphoffman/wip-rbd-du-validate

test/librbd: add test to verify diff_iterate size

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45221 from rzarzynski/wip-core-drop-unused-aiocompletion-in-librados

core, test: fix compile-time warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45415 from Matan-B/wip-matanb-watch-r-buffer

crimson/osd: fix osd crash due to buffer reference binding

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #45050 from hualongfeng/qatchange

common/compressor: fix the issue that cannot processed concurrently

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #44491 from nmshelke/fix-46075

ceph-fuse: ignore fuse mount failure if path is already mounted

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Fix a problem in store_test::BluestoreBrokenNoSharedBlobRepairTest where the check for active null-fm was wrong and so reporting bogus errors  when null-fm was inactive
The check need to access dynamic value and not config setting (which can be overridden)
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>

mgr/dashboard/monitoring: update grafana version

Fixes: https://tracker.ceph.com/issues/54311
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #45406 from wxypro/fix-typos

docs,options: fix typos

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45525 from idryomov/wip-rbd-trash-purge-schedule-relax-assert

qa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #45517 from liu-chunmei/crimson-fix-cmpxattr

crimson: let read operation return result in MOSDOpReply

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

python-common/drive_group: add extra_container_args to supported features

Should have been added when extending extra container args
to all the services but was missed

Signed-off-by: Adam King <adking@redhat.com>