qa: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

docs: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output. This field only exists if clones are
in pending or in progress state.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

mgr/volumes: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

Merge pull request #46472 from lxbsz/wip-55331

mds: notify the xattr_version to replica MDSes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #46575 from batrick/i51332

qa: exercise metadata replication in workloads

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #46919 from jsoref/spelling-docs

doc: Fix many spelling errors

Merge pull request #46602 from batrick/i55980

mds,qa: some balancer debug messages (<=5) not printed when debug_mds is >=5

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #46770 from rzarzynski/wip-crimson-setallochint

crimson: bring alloc hints to OpsExecuter, ignore them in SeaStore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #46929 from rzarzynski/wip-crimson-ftbfs-segment_cleaner

crimson/tools: fix FTBFS due to seastore/segment_cleaner.h

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #46087 from rhcs-dashboard/update-centralized-logging-docs

doc: update docs for centralized logging

Merge pull request #46942 from Schweinepriester/patch-1

README.md: HTTP => HTTPS

README.md: HTTP => HTTPS

Switching the link http://ceph.com/ from HTTP to HTTPS, so https://ceph.com/, to skip the redirect when opening it.
(http://ceph.com/ is being redirected to https://ceph.com/)

Signed-off-by: Kai Hollberg <kai.hollberg@googlemail.com>

doc: Fix many spelling errors

* administrators
* allocated
* allowed
* approximate
* authenticate
* availability
* average
* behavior
* binaries
* bootstrap
* bootstrapping
* capacity
* cephadm
* clients
* combining
* command
* committed
* comparison
* compiled
* consequences
* continues
* convenience
* cookie
* crypto
* dashboard
* deduplication
* defaults
* delivered
* deployment
* describe
* directory
* documentation
* dynamic
* elimination
* entries
* expectancy
* explicit
* explicitly
* exporter
* github
* hard
* healthcheck
* heartbeat
* heavily
* http
* indices
* infrastructure
* inherit
* layout
* lexically
* likelihood
* logarithmic
* manually
* metadata
* minimization
* minimize
* object
* of
* operation
* opportunities
* overwrite
* prioritized
* recipe
* records
* requirements
* restructured
* running
* scalability
* second
* select
* significant
* specify
* subscription
* supported
* synonym
* throttle
* unpinning
* upgraded
* value
* version
* which
* with

Plus some line wrapping and additional edits...

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Merge pull request #42095 from thotz/rgwvaultdocupdate

doc/radosgw/vault: update the vault documentation

Merge pull request #46938 from zdover23/wip-doc-2022-07-03-dev-guide-irc-registration-instructions-2

doc/dev: add IRC registration instructions

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: add IRC registration instructions

This PR adds to the Developer Guide a link to the OFTC's IRC nick
registration instructions.

Merge pull request #46906 from zdover23/wip-doc-2022-06-30-devguide-basic-workflow-the-the-fix

doc/dev: s/the the/the/ in basic-workflow.rst

Merge pull request #43085 from thotz/updategocephadminops

doc/radosgw/adminops: add latest go ceph library to adminops doc

doc/radosgw: Resolving conflicts with #43128

Signed-off-by: Anthony D'Atri <anthony,.datri@gmail.com>

Merge branch 'main' into updategocephadminops

Merge pull request #43128 from lbausch/adminops_php_binding_libraries

doc/radosgw/adminops: add PHP binding libraries

Merge pull request #46784 from adk3798/backport-cephadm-label

script/ceph-backport.sh: add label entries for cephadm

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #46892 from rkachach/fix_issue_56415

mgr/cephadm: fixing scheduler consistent hashing

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44493 from zenomri/wip-omri-cephadm-jaeger

mgr/cephadm: Add jaeger-tracing services support

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #46916 from ceph/revert-actions

Revert GitHub actions to label and backport doc/releases PRs

crimson/tools: fix FTBFS due to seastore/segment_cleaner.h

```
In file included from ../src/crimson/tools/store_nbd/tm_driver.cc:4:
../src/crimson/tools/store_nbd/tm_driver.h:7:10: fatal error: crimson/os/seastore/segment_cleaner.h: No such file or directory
    7 | #include "crimson/os/seastore/segment_cleaner.h"
      |
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #46922 from cyx1231st/wip-seastore-onodetree-compare-oid

crimson/onode-staged-tree: expose compare method for ghobject_t

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #46860 from ronen-fr/wip-rf-good-grants

osd/scrub: late-arriving reservation grants are not an error

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>

crimson/onode-staged-tree: expose compare method for ghobject_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/osd: implement CEPH_OSD_OP_SETALLOCHINT in OpsExecuter

This commits brings support setting allocation hints to `OpsExecuter`.

What is important to note that `SETALLOCHINTS`, at the ops execution
layer, behaves basically like `TOUCH`, and thus should be ignored
(for now) at the object store layer to not miss the part constituted
by `PGBackend::maybe_create_new_object()`.

Fixes: https://tracker.ceph.com/issues/56115
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/os: ignore CEPH_OSD_OP_SETALLOCHINT in SeaStore

At the moment crimson ignores this operation at the layer
of ops execution. However, this handles only those alloc
hints that come from clients while `set_alloc_hint` can be
called from `ReplicatedRecoveryBackend::prep_push_target()`.
Likely this was the reason behind the following crash:

```
INFO  2022-06-20 11:03:38,952 [shard 0] osd - Entering state: Started/ReplicaActive/RepRecovering
ERROR 2022-06-20 11:03:39,002 [shard 0] seastore - SeaStore::_do_transaction_step: bad op 39
```

Fixes: https://tracker.ceph.com/issues/56115
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #46854 from rhcs-dashboard/update-cypress-9.6

mgr/dashboard: update cypress to 9.7.0

Merge pull request #46886 from xxhdx1985126/wip-56399

crimson/os/seastore/omap_manager: bug fix for https://tracker.ceph.com/issues/56399

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #46834 from liu-chunmei/crimson-do_transactions

crimson/seastore:  cleanup do transactions create onode logic

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #46627 from liu-chunmei/crimson-fix-pg-prefix

crimson/osd: fix pg prefix error

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/test: touch object before access it.

no automatically create onode in seastore.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

crimson/seastore: cleanup seastore do_transaction create onode logic

not always create onode when it doesn't exists

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

Revert ".github: Label doc/releases PRs so tibdex/backport@v2 can backport"

This reverts commit 24e38224350394198c9f873800048d25b8c6548e.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Revert ".github: Support for automatically creating backport PRs when labeled"

This reverts commit f5f9459fb22d4c832cac88abfc1b92ab7304ed51.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Revert ".github: Add labels while PR is open"

This reverts commit 605b457c9857b370d1e545bc938be1c177151eee.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #46049 from soumyakoduri/wip-skoduri-dbstore-build

rgw/dbstore: Fix build errors on centos9

Merge pull request #46869 from nmshelke/fix-56384

qa/workunits/cephtool: check_response didn't find erasure-code string

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46888 from kotreshhr/fix-stop.sh

stop.sh: Fix umount in stop.sh

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #46693 from rkachach/fix_issue_54399

doc/cephadm: enhancing daemon operations documentation

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #46887 from pritha-srivastava/wip-rgw-sts-teuthology-failure

rgw/qa: fix for teuthology failure related to role metadata sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46858 from idryomov/wip-rbd-deep-copy-progress

librbd: update progress for non-existent objects on deep-copy

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc/dev: s/the the/the/ in basic-workflow.rst

This PR does exactly what it says: it changes a "the the"
to a "the".

I'd like to thank Josh Soref for bringing this
to my attention.

mgr/dashboard: update cypress to 9.7.0

Fixes: https://tracker.ceph.com/issues/56426
Signed-off-by: Nizamudeen A <nia@redhat.com>

crimson/os/seastore/omap_manager: fix error when listing multiple omap tree inner node

Fixes: https://tracker.ceph.com/issues/56399
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

src/test/crimson/seastore: add unit test case for listing entries of omap trees higher than 3 layers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/os/seastore/omap_manager: better debug outputs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #46436 from cyx1231st/wip-seastore-generational-cleanning

crimson/os/seastore: implement generational GC

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd: fix pg prefix error

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

Merge pull request #46843 from jsoref/spelling-docs

doc/releases: Fixes spelling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #46881 from ivancich/wip-complete-op-spamming-log

rgw: reduce logging level in rgw_bucket_complete_op

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge pull request #46880 from thinko/patch-1

doc/cephadm: osd: minor fixes in OSD config examples

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #46894 from zdover23/wip-doc-2022-06-29-doc-index-link-to-basic-workflow

doc/index.rst: add link to Dev Guide basic workfl.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/cephadm: fixing E128 warning

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

doc/index.rst: add link to Dev Guide basic workfl.

This PR adds a link to the "Basic Workflow" section of the
Developer Guide on the landing page of docs.ceph.com.

This PR is meant to improve the documentation for developers
new to Ceph and to guide them to instructions that will allow
them to become full-fledged contributors to the Ceph project
as quickly as possible.

The "Basic Workflow" page of the Developer Guide contains
information that answers almost all of the questions that I had
about contributing to the Ceph project when I was new to it,
and I am finally acting on my long-held conviction that the
"Basic Workflow" page of the Developer Guide should have a more
prominent position in the documentation suite than it has had.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #46864 from rzarzynski/wip-crimson-gcc11

crimson: switch to GCC11

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

mgr/cephadm: fixing scheduler consistent hashing

Signed-off-by: Redouane Kachach <rkachach@redhat.com>

qa/workunits/cephtool: check_response didn't find erasure-code string

1. If data or metadata pool is already in-use by filesystem
then it is not allowed to reuse the same pool for another
filesystems.

2. Test is failing because above(1) restrictions/checks comes
before checking erasure-code pools. Hence test is failing
and not finding expected error string in output.

3. Proposed fix checks newly added error string instead of
'erasure-code'.

4. Also adding new tests to verify string 'erasure-code'
by passing --force option so that check for pools reuse(1)
will be skipped and check for 'erasure-code' will be hit.

Fixes: https://tracker.ceph.com/issues/56384
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>

stop.sh: Fix umount in stop.sh

Introduced-by: 8a4a432e8b6e41ff13c33698cc053079f6786aa2
Signed-off-by: Kotresh HR <khiremat@redhat.com>

crimson/os/seastore/segment_cleaner: increase avaliable ratio limit

Journal trimming may consume unexpected number of segments when the
available ratio limit is reached with user transactions blocked, causing
ceph_abort(). So increase the limit as a simple workaround.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: improve GC policies with modify-time

* record_header_t to store the average modify time for dirty extents.
* Drop tracking rewrite-time.
* Drop the last-modify field in extent_info_t.
* Maintain modify-time during rewriting.
* Introduce 3 GC policies: greedy, benefit, and cost-benefit.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/segment_cleaner: improve log information

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: misc cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: implement generational GC

Place extents into the dedicated RecordSubmitter by their data-category
and reclaimed-count. Segments of different data-category or
reclaimed-count should have different locality in the access patterns,
which is the foundation to form a desired bimodal distribution of
segment utilizations, so that GC can be more efficient.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

rgw/qa: fix for teuthology failure related to role metadata sync
in case of pub sub zone.

fixes: https://tracker.ceph.com/issues/56175

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #46885 from myoungwon/wip-rename-seg-cleaner

seastore: rename segment cleaner to async cleaner

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

seastore: rename segment cleaner to async cleaner

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

librbd: make ImageCopyRequest::send_next_object_copy() return void

Make send_object_copies() consistent with handle_object_copy() wrt
calling send_next_object_copy().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rgw: reduce logging level in rgw_bucket_complete_op

There are two instances where pedestrian logging happens at level 0
but should be at a much higher level, such as 20. Both are fixed.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

doc/cephadm: osd: minor fixes in config examples

- Corrected drive counts (8 to 10) and rotational flag use (not consistent with HDD) in the "Multiple OSD specs for a single host" example
- Adjusted vendor names in "Multiple hosts with the same disk layout" for consistency of examples

Signed-off-by: Alex Handy <thinko@redhat.com>

Merge pull request #46660 from pritha-srivastava/wip-rgw-user-policy-test-changes

Changes to integrate user-policy(iam) tests in teuthology.

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #46857 from zdover23/wip-doc-2022-06-27-pdonnelly-to-venky

doc/dev_guide: change CephFS lead to vshankar

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #46581 from linuxbox2/wip-admin-errortrans

rgw: return ENOENT for NO_SUCH_BUCKET in radosgw-admin bucket stats

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46846 from tchaikov/cmake-ccache

cmake: use CMAKE_<LANG>_COMPILER_LAUNCHER for configuring ccache

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46835 from CongMinYin/wip-install-pmdk

install-deps: install pmdk libraries

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

ceph.spec.in: use %enable_devtoolset11 to enable GTS-11

%enable_devtoolset11 redefines %___build_pre by appending
`source scl_source enable gcc-toolset-11` to it. `___build_pre` should
be able to populate this setting to both %build and %install. and hence
address the FTBFS where we need to use the tool chain from GTS-11.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 49459d92fec553246b35719e9defc0ad7582f701)

ceph.spec.in: enable toolset 11 also in install

This can be surprising but we actually compile things during
the `install` stage of `rpm-build`. The example is the pybind's
`setup.py` which builds `rados_dummy.c`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph.spec.in: move and undef _annotated_build only for rhels.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph.spec.in: use gcc-toolset-11 for building crimson

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

doc/dev_guide: change CephFS lead to vshankar

This makes an overdue change. Venky Shankar is the
CephFS lead, and has been for a while.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

osd/scrub: late-arriving reservation grants are not an error

... as, barring a bug, these are simply the successful grants
received after one replica had failed to secure the required
resources.

Fixes: https://tracker.ceph.com/issues/56400
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

doc: update docs for centralized logging

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #46754 from ideepika/wip-opentelemetry-submodule

.gitmodules: add opentelemetry-cpp as a submodule

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46829 from soumyakoduri/wip-skoduri-dbstore-fix

rgw/dbstore: Fix crash in delete_stale_objs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

librbd: update progress for non-existent objects on deep-copy

As a side effect of commit e5a21e904142 ("librbd: deep-copy image copy
state machine skips clean objects"), handle_object_copy() stopped being
called for non-existent objects.  This broke progress_object_no logic,
which expects to "see" all object numbers so that update_progress()
callback invocations can be ordered.  Currently update_progress() based
progress reporting gets stuck after encountering a hole in the image.

To fix, arrange for handle_object_copy() to be called for all object
numbers, even if ObjectCopyRequest isn't created.  Defer the extra call
to the image work queue to avoid locking issues.

Fixes: https://tracker.ceph.com/issues/56181
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #46673 from ljflores/wip-mempool-documentation

doc/dev: improve wording and grammar of mempool page

Merge pull request #46756 from tobias-urdin/mgr-influx-pool-name-keyerror

mgr: influx: catch KeyError when looking up pool name

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46732 from rzarzynski/wip-55355

msg: fix deadlock when handling existing but closed v2 connection

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #46628 from tchaikov/wip-msg-cleanup

msg, mount: cleanups

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>

Merge pull request #46571 from NitzanMordhai/wip-nitzan-add-pg-log-output

osd: add option to dump pg log to pg command

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: rename a series of pmem libraries to pmdk

At first, libpmem was the only library. Later, pmem related libraries
such as libpmemobj and libpmem2 were gradually added. These libraries
were also integrated into one named pmdk. So rename to pmdk.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>

doc/dev: improve wording and grammar of mempool page

Signed-off-by: Laura Flores <lflores@redhat.com>

doc/releases: Fix many spelling errors

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

Merge pull request #46727 from cfsnyder/wip-54172-fix-scrub-abort

osd/scrubber/pg_scrubber.cc: fix bug where scrub machine gets stuck
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #46785 from kamoltat/wip-ksirivad-fix-56151

src/mgr/DaemonServer.cc: fix typo in output gap >= max_pg_num_change
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46761 from pdvian/wip-autoscaler-adjust

pybind/mgr/autoscaler: Donot show NEW PG_NUM value if autoscaler is not on
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>