doc/mgr/telemetry: update default interval

Commit 712987d533 changed the default interval to 24h;
updating the docs to match (this also should go to
the Nautilus branch as that commit landed there too
in https://github.com/ceph/ceph/pull/27709)

Signed-off-by: Tim Serong <tserong@suse.com>

Merge pull request #30583 from mika/mika/typos

doc: fix typos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30614 from tchaikov/wip-no-more-osdmap-full

librados,osd,mon: remove traces of CEPH_OSDMAP_FULL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

PendingReleaseNotes: document full/nearfull changes

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30635 from tchaikov/wip-42079

mgr: use a struct for DaemonKey

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30699 from neha-ojha/wip-more-recovery-tests

qa/suites/rados/thrash: increase async and partial recovery test coverage

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30482 from pcuzner/k8sevents-remote-support

mgr/k8sevents: Add support for remote kubernetes

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/k8sevents: Add support for remote kubernetes

The initial implementation sent events to the
kubernetes cluster Ceph is running under managed
by rook-ceph. This patch extends this support to
include sending events to an external kubernetes
cluster, that may just be consuming ceph resources

Additional docs added to help people use the
module, either with or without rook-ceph.

It also address a bug where the module was loaded
outside of a rook-ceph environment.

Fixes: https://tracker.ceph.com/issues/41737
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

Merge PR #30692 into master

* refs/pull/30692/head:
osd/PeeringState: disable read lease until require_osd_release >= octopus

Reviewed-by: Samuel Just <sjust@redhat.com>

qa/suites/rados: add more-async-partial-recovery.yaml in the thrash suite

This will increase both async and partial recovery.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados: add more-partial-recovery.yaml in the thrash suite

Increasing osd_object_clean_region_max_num_intervals to track more
clean regions, resulting in more partial recovery.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados: add more-async-recovery.yaml in the thrash suite

Reducing osd_async_recovery_min_cost to 1 should allow more
asynchronous recovery to happen.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #30709 into master

* refs/pull/30709/head:
msg/async: rename outcoming_bl -> outgoing_bl in AsyncConnection.

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async: rename outcoming_bl -> outgoing_bl in AsyncConnection.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #29711 into master

* refs/pull/29711/head:
doc: Update documentation for LazyIO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #30473 into master

* refs/pull/30473/head:
doc: Fix FUSE expansion

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30251 into master

* refs/pull/30251/head:
doc: Update requirements for using CephFS

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: Update requirements for using CephFS

Fixes: https://tracker.ceph.com/issues/41470
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #30654 from ajarr/fix-42096

mgr/volumes: fix incorrect snapshot path creation

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30627 into master

* refs/pull/30627/head:
qa/suites/rados/verify: whitelist MON_DOWN when using valgrind
qa/suits/rados/basic/tasks/rados_api_tests: pgs can go degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #30640 into master

* refs/pull/30640/head:
osd/PrimaryLogPG: remove unused reply creation path
osd/PrimaryLogPG: include op_returns in dup replies
osd/PrimaryLog: drop unused reply_ctx() variant
osd/PrimaryLogPG: remove dead already_ack()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #30596 into master

* refs/pull/30596/head:
qa/suites/rados/singleton/all/ec-lost-unfound: no rbd pool

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30675 from idryomov/wip-set-kernel-secret

common/secret.c: don't pass uninitialized stack data to the kernel

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

osd/PeeringState: disable read lease until require_osd_release >= octopus

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Update npm packages (#30439)

mgr/dashboard: Update npm packages

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #30626 into master

* refs/pull/30626/head:
doc: document recover_session= option in mount.ceph manpage
doc: update documents on using kcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30542 into master

* refs/pull/30542/head:
doc: cleanup CephFS Landing Page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Show smartctl summary in OSD tab (#30378)

mgr/dashboard: Show smartctl summary in OSD tab

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29922 from smithfarm/wip-41524

cmake: set empty RPATH for some test executables

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #30416 into master

* refs/pull/30416/head:
client: add procession of SEEK_HOLE and SEEK_DATA in lseek.

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30653 from tchaikov/wip-os/drop-kinetic

os: remove KineticStore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28569 from alimaredia/wip-rgw-ioctx-operate-replace

rgw: replace direct calls to ioctx.operate()

doc: document recover_session= option in mount.ceph manpage

Fixes: https://tracker.ceph.com/issues/40957
Signed-off-by: Jeff Layton <jlayton@redhat.com>

common/secret.c: don't pass uninitialized stack data to the kernel

ceph_unarmor() returns the number of bytes decoded, which can be (and
usually is) smaller than the size of the payload array.

set_kernel_secret() has behaved this way ever since it was introduced
in commit bee85518e288 ("mount.ceph: Use kernel key management API when
possible.").  The reason it didn't cause problems in the kernel is that
the encoding includes the actual length of the secret and there is no
check for the end of the supplied payload (see ceph_key_preparse() in
net/ceph/crypto.c).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #30655 from tchaikov/wip-cmake/crimson/pass-args

cmake: pass arguments to crimson tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

client: add procession of SEEK_HOLE and SEEK_DATA in lseek.

Fixes: https://tracker.ceph.com/issues/42107
Signed-off-by: Shen Hang <harryshen18@gmail.com>

Merge pull request #30613 from joke-lee/rgw-ceph-dencoder-add-RGWPeriodLatestEpochInfo

ceph-dencoder: add RGWPeriodLatestEpochInfo support.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Update npm packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

doc: update documents on using kcephfs

Overhaul the fstab.rst and kernel.rst files with more relevant examples
now that the mount helper does autodiscovery. Add a preamble to
kernel-features.rst and add a warning about the deprecation of inline
data.

Fixes: https://tracker.ceph.com/issues/42025
Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/volumes: fix incorrect snapshot path creation

When constructing the snapshot path, the components are a mix of
byte and string type objects. The 'os.path.join()' method needs
the components to be of the same type. Hence convert all components
to byte type objects.

Introduced-by: 3d63cd947f55a8508f972ee249ea6a04836429ae
Fixes: https://tracker.ceph.com/issues/42096
Signed-off-by: Ramana Raja <rraja@redhat.com>

cmake: pass arguments to crimson tests

otherwise they will take all CPU cores and all memory

Signed-off-by: Kefu Chai <kchai@redhat.com>

build/ops: set empty RPATH for some test executables

Fixes: https://tracker.ceph.com/issues/41524
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30587 from sebastian-philipp/python-common-readme

python-common: Add small Readme

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #30341 into master

* refs/pull/30341/head:
cephfs-shell: better complain info, when deleting non-empty directory

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge PR #30559 into master

* refs/pull/30559/head:
mds: Reorganize class members in MDBalancer header

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #30580 into master

* refs/pull/30580/head:
client: don't ceph_abort on bad llseek whence value
client: remove Inode dir_contacts field

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #30407 from joscollin/wip-B41841-yes-really-mean-it

mgr/volumes: protection for `fs volume rm` command

doc: cleanup CephFS Landing Page

Fixes: https://tracker.ceph.com/issues/41952
Signed-off-by: Milind Changire <mchangir@redhat.com>

os: remove KineticStore

kinetic-cpp-client and kinetic project as a whole are not acitvely
supported or maintained anymore. see

- https://github.com/Kinetic/kinetic-cpp-client
- http://www.openkinetic.org

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #30552 into master

* refs/pull/30552/head:
cmake: fix libtsan detection
test: librados startup/shutdown racer test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30630 from tchaikov/wip-build-on-centos8

install-deps,rpm,do_cmake: build on RHEL/CentOS 8

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #30531 from alimaredia/wip-rgw-maven-version-update

qa: bump maven repo version in s3a_hadoop.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30615 from xiexingguo/wip-progress-time-remaining

mgr/progress: estimated remaining time for events

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30591 from xiexingguo/wip-balancer-throttler

mgr/balancer: upmap_max_iterations -> upmap_max_optimizations; behave as it is per pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29674 from athanatos/sjust/wip-fio-trace

os/bluestore: expand lttng tracepoints, improve fio_ceph_objectstore backend

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cls: CLS_CLIET_HIDE_IOCTX changes in cls headers

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: various ioctx.operate refactoring changes

- remove unused #include for cls_client headers
- refactor cls_timeindex_trim call in
rgw_object_expirer_core.cc
- add more CLS_CLIENT_HIDE_IOCTX in header files
- move target_compile_definitions to rgw_a target
- make repeat cls helper functions static

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: overload cls_rgw functions that do io_ctx.operate()

Call the new overloaded functions and call
rgw_rados_operate after them.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

cls: remove unused cls_rgw_reshard_get_head()

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: replace ioctx.operate() with rgw_rados_operate()

For all operate calls in:
rgw_op.cc
rgw_orphan.cc
rgw_reshard.cc
rgw_rados.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

cmake: fix libtsan detection

This variable is case-sensitive.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

osd/PrimaryLogPG: remove unused reply creation path

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: include op_returns in dup replies

We are storing the return metadata; actually use it when sending dup
replies!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30620 from votdev/issue_42069_validate_bucket_name

mgr/dashboard: Bucket names cannot be formatted as IP address

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kai Wagner <kwagner@suse.com>

Merge pull request #30647 from s0nea/wip-dashboard-fix-typo

mgr/dashboard: fix small typos in description message

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29925 from sebastian-philipp/run_tox-overwrite-build-dir

script/run_tox.sh: Don't overwrite the build dir

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Fix FUSE expansion

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

mgr/dashboard: fix two small typos

Remove unnecessary blanks.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

script/run_tox.sh: Don't overwrite the Ceph's bulid dir

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #30639 from tchaikov/wip-crimson-readable

crimson/osd: implement readable/lease related methods

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30644 from majianpeng/osd-remove-unused-func

osd: remove unused function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30632 from xiexingguo/wip-kick-recovery-pri

osd/PrimaryLogPG: always use strict priority ordering for kicked recovery ops

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30642 from Rjerk/wip-fix-doc

doc/radosgw: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: add closing ")" in log

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd/pg: implement schedule_renew_lease()

see also d883db70282c406edaceb72a90ecddccd97a0161

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implement ShardServices::get_hb_stamps()

PeeringState::proc_lease() requires a valid hb_stamps[0], which is in
turned maintained by `pl`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd/pg: implement queue_check_readable() and recheck_readable()

see also 7aec060e0aff4588ff51f744dd194d1c0f7793c2

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_time: do not handle negative case

this change partially reverts 353a0e5f, unlike `signedspan`,
`timespan::rep` is unsigned. so no need to handle negative case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Bucket names cannot be formatted as IP address

In general, bucket names should follow domain name constraints:
- Bucket names must be unique.
- Bucket names cannot be formatted as IP address.
- Bucket names can be between 3 and 63 characters long.
- Bucket names must not contain uppercase characters or underscores.
- Bucket names must start with a lowercase letter or number.
- Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.). Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number.

Fixes: https://tracker.ceph.com/issues/42069
Signed-off-by: Volker Theile <vtheile@suse.com>

osd: remove unused function

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc/man: update ceph(8) to drop "full" flag

`ceph {set,unset} full` does not take effect since mimic, and monitor
does not support this command since Octopus. so let's document this
change.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #30396 into master

* refs/pull/30396/head:
doc: document mds journaling

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30463 into master

* refs/pull/30463/head:
vstart_runner: allow the use of it with kernel mounts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/radosgw: fix typo

Signed-off-by: Liu Lan <liulan@umcloud.com>

mgr: use a struct for DaemonKey

instead of using `std::pair<>` for presenting it. there are three
advantages:

1. better names for readability: `type` and `name` are better names
   than `first` and `second`
2. so we can specliaze its behavior by its type: define
   `operator<<` once, and use it everywhere. no need to worry
   about ADL to pickup the generic operator of `operator<< (..., pair<>)`
   anymore. so we can safely use `<< key`, and no need to
   use `<< key.first << '.' << key.second`. kill the printed form of
   `osd,1` once and for all, all of them are now "osd.1".
3. consolidate the print/parse in a single place

Fixes: https://tracker.ceph.com/issues/42079
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/progress: estimated remaining time for events

Fixes: https://tracker.ceph.com/issues/40419
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30638 from uweigand/cmake-atomic-16byte

cmake: Test for 16-byte atomic support on IBM Z

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLog: drop unused reply_ctx() variant

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: remove dead already_ack()

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/verify: whitelist MON_DOWN when using valgrind

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: Test for 16-byte atomic support on IBM Z

On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.

Instead, Boost uses 16-byte atomics.  This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic.  The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.

Fixed by adding a test for 16-byte atomic support on IBM Z.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>

Merge pull request #30617 from xiexingguo/wip-weird-daemon-key

mgr: fix weird health-alert daemon key

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: fix weird health-alert daemon key

Was:
  19 slow ops, oldest one blocked for 34 sec, daemons [osd,2,osd,4] have slow ops.

Now:
  153 slow ops, oldest one blocked for 38 sec, daemons [osd.3,osd.4,osd.5] have slow ops.

Fixes: https://tracker.ceph.com/issues/42079
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #29236 from liewegas/wip-read-hole-bypg

osd: implement per-pg leases to avoid stale reads

Reviewed-by: Samuel Just <sjust@redhat.com>

osd/PrimaryLogPG: always use strict priority ordering for kicked recovery ops

See 590c4979d40ea2073da8180140303ac067d87cc4 for the background of
this change.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30527 from uweigand/bloom-filter-endian

common/bloom_filter: Fix endian issues

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30616 from ifed01/wip-ifed-fix-fsck

os/bluestore: fix invalid stray shared blob detection in fsck.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30625 from tchaikov/wip-42087

mgr/BaseMgrStandbyModule: drop GIL in ceph_get_module_option()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Revert "osd: give recovery ops initialized by client op a higher priority"

This reverts commit c0f87e0f91539d4d12dad5bddf95a97457a21c67.

The 'osd_op_queue_cut_off' config option determines which level of
high priority ops should use strict priority ordering and may change
from time to time. Since the main strategy of 'osd_kick_recovery_op_priority'
is to simply follow up 'osd_op_queue_cut_off', we can instead make a direct
use of 'osd_op_queue_cut_off' to achieve the same thing explicitly.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

ceph-dencoder: add RGWPeriodLatestEpochInfo support.

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>