qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3

To workaround the libarchive dependency issue seen with centos 8, which
has been causing consistent failures like

```
2021-06-04T04:52:51.147 INFO:tasks.workunit.client.0.smithi071.stdout:Installed:
2021-06-04T04:52:51.148 INFO:tasks.workunit.client.0.smithi071.stdout:  cmake-3.18.2-9.el8.x86_64                 cmake-data-3.18.2-9.el8.noarch
...
2021-06-04T04:52:57.554 INFO:tasks.workunit.client.0.smithi071.stderr:+ cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_TESTS=ON -DWITH_LIBRADOS=ON -DWITH_SNAPPY=ON -DWITH_GFLAGS=OFF -DFAIL_ON_WARNINGS=OFF ..
2021-06-04T04:52:57.579 DEBUG:teuthology.orchestra.run:got remote process result: 127
2021-06-04T04:52:57.580 INFO:tasks.workunit.client.0.smithi071.stderr:cmake: symbol lookup error: cmake: undefined symbol: archive_write_add_filter_zstd
```
More details in https://tracker.ceph.com/issues/51101#note-5

Fixes: https://tracker.ceph.com/issues/51101
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 345cb641ed63e5b88f799a0b50bddb4028ed2589)

Merge pull request #42258 from sseshasa/wip-51603-pacific

pacific: qa/standalone: Add missing cleanups after completion of a subset of osd and scrub tests.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42231 from cfsnyder/wip-51081-pacific

pacific: rgw: parse tenant name out of rgwx-bucket-instance

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42230 from cfsnyder/wip-51329-pacific

pacific: rgw: avoid infinite loop when deleting a bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42233 from smithfarm/wip-51579-pacific

pacific: common/Formatter: include used header

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42221 from neha-ojha/wip-50393-pacific

pacific: qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of proc.communicate()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42165 from batrick/i51500

pacific: qa: FileNotFoundError: [Errno 2] No such file or directory: '/sys/kernel/debug/ceph/3fab6bea-f243-47a4-a956-8c03a62b61b5.client4721/mds_sessions'

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #42088 from lxbsz/wip-51285

pacific: mds: to print the unknow type value

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #42086 from kotreshhr/wip-51200-pacific

pacific: mgr/volumes: Add config to insert delay at the beginning of the clone

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #39834 from sebastian-philipp/pacific-backport-39131-39373

pacific: mgr/rook: Add timezone info

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #42316 from rhcs-dashboard/wip-51657-pacific

pacific: mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'

Fixes: https://tracker.ceph.com/issues/51643
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Removing the usage of rgw_frontend_ssl_key from the rgw service form.

(cherry picked from commit f6afed5aa51d80257b4883c976bbae7c3b5cc524)

Merge pull request #42264 from yuriw/wip-yuriw-p2p-pacific

qa/tests: advanced pacific version to reflect the latest 16.2.5 point

Merge pull request #40694 from singuliere/wip-50209-pacific

pacific: qa/rgw: move ignore-pg-availability.yaml out of suites/rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42093 from thotz/rgwkmsvaultsslsupport-pacific

pacific: rgw: extending existing ssl support for vault KMS

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #42265 from neha-ojha/wip-rn-cleanup

pacific: doc/releases/pacific.rst: remove notes about autoscaler

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

qa/tests: advanced pacific version to reflect the latest 16.2.5 point

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc/releases/pacific.rst: remove notes about autoscaler

This was reverted in 504d0cb120f02b1940f81505bd37a54773625af0, but
got added back by mistake in 6a3dde8b498af5e33242955a0609b0e27adf4877.

The relevant code in master was not reverted and is being reworked, we'll
update PendingReleaseNotes in master after that.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/standalone: Add missing teardowns to a subset of osd-scrub-repair tests

Tests identified with missing teardown within osd-scrub-repair.sh:
  1. TEST_periodic_scrub_replicated()
  2. TEST_scrub_warning()
  3. TEST_request_scrub_priority()

Centralize setup and teardown within the run() function for all the tests.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 84cab65e3aafcf656b860abd72a09799458f6bc5)

 Conflicts:
qa/standalone/scrub/osd-scrub-repair.sh
 - Removed non-existent test TEST_auto_repair_bluestore_tag() and its
   associated helper functions initiate_and_fetch_state() and
   wait_end_of_scrub()

qa/standalone: Add missing teardowns to a subset of osd tests

The following files and tests in them did not teardown the
cluster after a test completed.

1. osd/osd-force-create.sh
2. osd/osd-reuse-id.sh
3. osd/pg-split-merge.sh

This wouldn't cause issues if the tests are run individually. But when
running all the tests in the files mentioned above, it could introduce
unexpected test failures down the line. For e.g., multiple tests may
create pools with same name and if they are not cleaned up properly, this
could result in unexpected failures in a subsequent test.

Fixes: https://tracker.ceph.com/issues/51580
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit a96c34f0ee91af55f80fb58936299f5effc0c0cc)

Merge pull request #42250 from paulreece42/pacific

pacific: doc/rados/operations: s/max_misplaced/target_max_misplaced_ratio/

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40692 from singuliere/wip-50240-pacific

pacific: Warning Cleanup and Clang Compile Fix

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

doc: added documentation on additional throttling options for the PG balancer module

Signed-off-by: Paul Reece <paul@reece.cc>
(cherry picked from commit eafa979b7972be72f7bde1dd29f40b403f29a496)

doc/rados/operations: s/max_misplaced/target_max_misplaced_ratio/

max_misplaced with replaced by in target_max_misplaced_ratio
edbd592ee44e02a5328e1510879555c2f9dcfc9e, but the document was not
sync'ed. let's update it accordingly.

Fixes: https://tracker.ceph.com/issues/50745
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc43462ab3d4b38cf0c496ae084b82c20ba449ae)

16.2.5

include/denc: include used header

to address the FTBFS like:

/home/kchai/ceph/src/include/denc.h: In member function ‘void DencDumper<T>::dump() const’:
/home/kchai/ceph/src/include/denc.h:121:60: error: ‘O_BINARY’ was not declared in this scope
     int fd = ::open(fn, O_WRONLY|O_TRUNC|O_CREAT|O_CLOEXEC|O_BINARY, 0644);
                                                            ^~~~~~~~
/home/kchai/ceph/src/include/denc.h:121:60: note: the macro ‘O_BINARY’ had not yet been defined
In file included from /home/kchai/ceph/src/include/statlite.h:14,
                 from /home/kchai/ceph/src/include/types.h:41,
                 from /home/kchai/ceph/src/auth/Crypto.h:19,
                 from /home/kchai/ceph/src/auth/Crypto.cc:21:

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 03770f75562812c683ff1503d2032dcd66d83289)

mon,osd: always init local variable

this change silences the warning like:

../src/mon/Monitor.cc: In member function ‘void Monitor::handle_command(MonOpRequestRef)’:
../src/mon/Monitor.cc:3703:55: warning: ‘osd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 3703 |     uint64_t seq = mgrstatmon()->get_last_osd_stat_seq(osd);
      |                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7c2f58ea5ec70e5606ab362e6f4cfda087cec65a)

common/Formatter: include used header

otherwise it fails to build with gcc-toolset-10, like:

../src/common/Formatter.cc: In member function ‘virtual void ceph::XMLFormatter::close_section()’:
../src/common/Formatter.cc:449:8: error: ‘transform’ is not a member of ‘std’
  449 |   std::transform(section.begin(), section.end(), section.begin(),
      |        ^~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ff79cae5a1243111c8a039426081638d40a258fa)

rgw: parse tenant name out of rgwx-bucket-instance

used by multisite bucket full sync to request the listing of a specific
bucket instance. if the bucket lives under a tenant, we need to get that
out of the rgwx-bucket-instance header, because the http request path
only names the bucket

Fixes: https://tracker.ceph.com/issues/50785
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 291342425e4b49de9b6985c718f6cb9210f5554d)

rgw: avoid infinite loop when deleting a bucket

When deleting a bucket with an incomplete multipart upload that
has about 2000 parts uploaded, we noticed an infinite loop, which
stopped s3cmd from deleting the bucket forever.
Per check, when the bucket index was sharded (for example 128
shards), the original logic in
RGWRados::cls_bucket_list_unordered() did not calculate
the bucket shard ID correctly when the index key of a data
part was taken as the marker.

The issue is not necessarily reproduced each time. It will depend
on the key of the object. To reproduce it in 128-shard bucket,
we use 334 as the key for the incomplete multipart upload,
which will be located in Shard 127 (known by experiment). In this
setup, the original logic will usually come out a shard ID smaller
than 127 (since 127 is the largest one) from the marker and
thus a circle is constructed, which results in an infinite loop.

PS: Some times the bucket ID calculation may incorrectly going forward
instead of backward. Thus, the check logic may skip some shards,
which may have regular keys. In such scenarios, some non-empty buckets may
be deleted by accident.

Fixes: http://tracker.ceph.com/issues/49206
Signed-off-by: Jeegn Chen <jeegnchen@tencent.com>
(cherry picked from commit 3cafe5774a5a453d58a3a6bed1f02d3200c4bb1d)

Merge pull request #41774 from trociny/wip-51143-pacific

pacific: cls/rgw: look for plain entries in non-ascii plain namespace too

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of proc.communicate()

the loop of proc.communicate() on python3.6, where we always are able to
get something out of stdout and/or stderr PIPEs. and the `stdout` and
`stderr` keep growing until out of memory. and teuthology considers
the command crashed after a while.

Fixes: https://tracker.ceph.com/issues/50393
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f2e0770c276e6fffa01a0bcb2d969f02ee1aafef)

Merge pull request #42153 from rhcs-dashboard/wip-51478-pacific

pacific: mgr/dashboard: fix OSD out count

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: huww98 <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #42156 from rhcs-dashboard/wip-51489-pacific

pacific: mgr/dashboard: run cephadm-backend e2e tests with KCLI

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

qa: use kclient xattr to lookup client id

Added relatively recently but in RHEL 8.4.

Fixes: https://tracker.ceph.com/issues/51183
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f92a4f4330da5f7d1d277ba6744f819c4c541eb4)

qa: refactor reading debug file code

No need to invoke Python to read a file!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 891c2773e9067b0d40d222f3f69635211809efaa)

qa: get mount id before failing fs

Otherwise getxattr will hang.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 97a86453f87f8e731bd3cce6bc525b412bf68cf3)

Merge PR #42159 into pacific

* refs/pull/42159/head:
pybind/ceph_volume_client: use cephfs mkdirs api

Reviewed-by: Ramana Raja <rraja@redhat.com>

pybind/ceph_volume_client: use cephfs mkdirs api

This _mkdir_p should never have worked as the first directory it tries
to stat/mkdir is "", the empty string. This causes an assertion in the
client. I'm not sure how this code ever functioned without causing
faults. They look like:

    2021-07-01 02:15:04.449 7f7612b5ab80  3 client.178735 statx enter (relpath  want 2047)

The assertion is caused by a C++ exception:

/usr/include/c++/8/string_view:172: constexpr const _CharT& std::basic_string_view<_CharT, _Traits>::operator[](std::basic_string_view<_CharT, _Traits>::size_type) const [with _CharT = char$_Traits = std::char_traits<char>; std::basic_string_view<_CharT, _Traits>::size_type = long unsigned int]: Assertion '__pos < this->_M_len' failed.
        Aborted (core dumped)

Where relpath is just the path passed to Client::stat.

This commit only applies to Pacific and older because master no longer
has this library.

Fixes: https://tracker.ceph.com/issues/51492
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: run cephadm-backend e2e tests with KCLI

Fixes: https://tracker.ceph.com/issues/51300
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 5c03b49c4da55cf8d0c679ecb2c58182e4d3361a)

mgr/dashboard: fix OSD out count

Think we have 3 OSDs out but up (prepare for re-formatting to change min_alloc_size), and another OSD down but in
(during reboot). The dashboard will display "1 down, 2 out", which is obviously incorrect. It should be "1 down, 3 out"

Fixes: https://tracker.ceph.com/issues/51376
Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit 921b48234777eba1a15c126f964cc47c0932879a)

Merge pull request #42140 from rhcs-dashboard/wip-51475-pacific

pacific: mgr/dashboard: User database migration has been cut out

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #42120 from neha-ojha/wip-51453-pacific

pacific: qa/suites/rados: add simultaneous scrubs to the thrasher

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #42065 from batrick/i51232

pacific: qa: scrub code does not join scrubopts with comma

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42067 from batrick/i51251

pacific: qa: fs:upgrade uses teuthology default distro

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: User database migration has been cut out

This PR will revert changes done by https://tracker.ceph.com/issues/49645 to auto-migrate user database v1 to v2.

Fixes: https://tracker.ceph.com/issues/51443
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 82922811612813840a684af993ca7c95a0b48d53)

Merge pull request #41842 from Daniel-Pivonka/pacific-backport-41740

pacific: mgr/nfs: do not depend on cephadm.utils

Merge pull request #42117 from liewegas/cephadm-june-last-pacific

pacific: cephadm june final batch

Merge pull request #42119 from rhcs-dashboard/wip-51447-pacific

pacific: mgr/dashboard: Fix bucket name input allowing space in the value

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #42061 from batrick/i50913

pacific: MDS heartbeat timed out between during executing MDCache::start_files_to_recover()

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42068 from batrick/i51286

pacific: MDSMonitor: crash when attempting to mount cephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42097 from vshankar/wip-51413

pacific: cephfs-mirror: reopen logs on SIGHUP

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42072 from batrick/i51414

pacific: mds: "FAILED ceph_assert(r == 0 || r == -2)"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42071 from batrick/i51412

pacific: mds: mkdir on ephemerally pinned directory sometimes blocked on journal flush

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42070 from batrick/i51324

pacific: pacific: client: abort after MDS blocklist

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42069 from batrick/i51322

pacific: qa: test_data_scan.TestDataScan.test_pg_files AssertionError: Items in the second set but not the first:

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42066 from batrick/i51235

pacific: doc: pacific updates

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42064 from batrick/i51231

pacific: pybind/mgr/snap_schedule: Invalid command: Unexpected argument 'fs=cephfs'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42063 from batrick/i51230

pacific: qa: fs:bugs does not specify distro

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

pacific: qa: add missing package exclusions

ceph-immutable-object-cache does not exist on nautilus.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/pybind/test_ceph_argparse: add a test where args contains comma

to ensure the support for the new syntax of "prefix --name bon,jour"
does not break existing behavior of "prefix hello cruel,world" where value2
contains reads "hello,world", and the parsed result should be

prefix="prefix"
value=["hello", "cruel,world"]

instead of something like

prefix="prefix"
value=["hello", "cruel", "world"]

or

prefix="prefix"
value=["cruel", "world"]

the above test only applies to the test where "value" is a CephString.
if "value" is a CephChoices, the parsed argument should be

prefix="prefix"
value=["hello", "cruel", "world"]

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9b6be59d6fa6ff7f6c5d773b6774728cf828b768)

cmake: run nose test using "python3 -m nose"

otherwise the tests are not performed at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 49be76ce11085bd333b5b8b281d8676c5ed450c2)

test/pybind/test_ceph_argparse: check for expected argument

instead of comparing the validated argument with empty dict, check for
its content for better coverage of testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cf4570d663d9c0fb656a54adc03fdfe83ead6ed0)

test/pybind/test_ceph_argparse: decorate helper method with @nottest

otherwise nose tries to run it as a test as its name contains "test", like

E.
======================================================================
ERROR: Runs validate in different arg/kargs ways.
----------------------------------------------------------------------
TypeError: arg_kwarg_test() missing 3 required positional arguments: 'prefix', 'args', and 'sig'

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3d7516f8daefcefa06a46981a62442d9193164f6)

test/pybind: do not test obsoleted command

"scrub" command was marked obsoleted in
e9a5ce0897efc6126caeebea9900bf05ec3d2174, so the test_ceph_argparse
cannot retrieve its command description using "get_command_descriptions"
cli tool anymore, let's drop the related test accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 77cb1fed0c30eec83ee0e4e0899dfdc1d4c5cc51)

test/pybind: drop forward and readforward from test

these two write modes were dropped in
a0a3ed324a130a9934735927600f01d1fa65bc0d, so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b8389557ad6297256f6a793723a5b601de739f2d)

test/pybind: do not use tab for indent

the python source file is idented using spaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 952285dcde501b38a63533768705526d3c5f02dc)

pybind/ceph_argparse: validate csv if desc.N

if desc.N is not None, we should take the argument as a comma separated
values, and validate the values individually.

restructure the validate() function and its helpers to pass the
validated args if desc.N explicitly, as desc.instance.val should only
hold a single value of desc.instance type, otherwise we need to reset
it after collecting all the argument in a CSV string value is parsed.

this change would be less consistent and hackish.

and update a test to verify this behavior.

Fixes: https://tracker.ceph.com/issues/51145
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0948a78ee6524911bd1fb76f1bf285e90e3cd349)

test/pybind: pep8 cleanups

* do not import unused symbols
* add two empty lines before method
* aligned indents

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc182888a3ca557e330e4317e0c50c1e80dbde4d)

Merge pull request #42045 from tchaikov/pacific-51354

pacific: osd/osd_type: use f->dump_unsigned() when appropriate

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>

Merge pull request #42019 from neha-ojha/wip-50791-pacific

pacific: osd: clear data digest when write_trunc

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41944 from ronen-fr/wip-ronenf-pac-50346

pacific: osd/scrub: replace a ceph_assert() with a test

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42062 from batrick/i51203

pacific: mds: CephFS kclient gets stuck when getattr() on a certain file

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42060 from batrick/i50875

pacific: mds: MDSLog::journaler pointer maybe crash with use-after-free

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42059 from batrick/i50848

pacific: mds: "cluster [ERR]   Error recovering journal 0x203: (2) No such file or directory" in cluster log"

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42058 from batrick/i50846

pacific: mds: journal recovery thread is possibly asserting with mds_lock not locked

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42057 from batrick/i50636

pacific: session dump includes completed_requests twice, once as an integer and once as a list

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #42056 from batrick/i50630

pacific: mds: Error ENOSYS: mds.a started profiler

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #41460 from idryomov/wip-qa-focal-pacific

pacific: qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41052 from lxbsz/wip-50445-pacific

pacific: client: make Inode to inherit from RefCountedObject

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #40856 from batrick/i50282

pacific: MDS slow request lookupino #0x100 on rank 1 block forever on dispatched

Reviewed-by: Jeff Layton <jlayton@redhat.com>

mgr: move GIL helpers to Gil.{h,cc}

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 88db7b190a974d70c08a1951b06b7fc9957b1b60)

qa/suites/rados: add simultaneous scrubs (multiple options) to the thrasher

Setting osd-max-scrubs to either 2 or 3.

Triggered by https://tracker.ceph.com/issues/50346

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit d232c4e8d8e4ca0ff428960235b61ea398bbec07)

mgr/dashboard: Fix bucket name input allows space in the value

Fixes: https://tracker.ceph.com/issues/51368
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 2e94611a65c26f1db79d535f2a32e8bc747ecdd7)

mgr/ActivePyModules: streamline mgr_ips a bit

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 49d5a5bdea2ca925ee5030d19d3793927c2a1fbd)

mgr/ActivePyModules: don't take unnecessary lock

This isn't needed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ee6f97b3ed172be98dd429a4aef54de5fb689f7)

pybind/mgr/mgr_module: use get("mgr_ips") for standby get_mgr_ip()

This is necessary to allow the standby mgr to bind to the right IP.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d159adc20e005dc7f7a7cea06c2ee7aa6f06d559)

mgr: add get() for standby modules

Fixes: https://tracker.ceph.com/issues/51446
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8b8e7522982476d25685ad055d4a89f6173041f8)

doc/cephadm: improving "Monitoring the Upgrade"

This PR improves the section "Monitoring the Upgrade"
in the "Upgrading Ceph" chapter of the cephadm documentation.

This PR introduces a couple of section breaks with signposting
information in their titles, and rewrites some sentences in order
to reduce the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 2dcdef0d43fb269447b5a00c8c29dc508919768d)

doc/cephadm: improve "Upgrading Ceph" (main)

This PR makes a couple of minor improvements to the text under the
top-level section "Upgrading Ceph" in the "Upgrading Ceph" chapter of
the cephadm documentation.

This one, mercifully, contains only a couple of changes.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit b4bfb3efb6d61cc19b286bb6a43c4d7db643df59)

cephadm: Fix normalize_image_digest for local registries

Cause they typically don't have dots in it.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ef5657cf6aa3558a20516cfa1a793dfba513db4)
 - conflict with adjacent test for the journald logging, which was not backported

doc/cephadm: enrich "Disabling Automatic Deploy..."

This PR rewrites and reformats the section "Disabling Automatic
Deployment of Daemons" in the "Service Management" chapter of the
cephadm guide.

I've rewritten some sentences, removed some "please"s, and added
some section titles so that the content in this is better
signposted.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit fee80411297589f80ff458bd20902e547e41c603)

doc/cephadm: enrich "deployment of daemons"

service-management.rst contained a section called
"Deployment of Daemons" that needed a rewrite by
a native English speaker.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 14dfbff2cd844885ab521de944ebeb0bb92bb72d)

src/vstart: create rgw user with system flag

As dashboard requires the system flag to be enabled.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 0d0cd4c6b0e8bbeaa71f54c831d67ef880e58016)

src/vstart: deploy rgw service with cephadm

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 63f5fa2e5e8946be4b4123c588bb133f1b18412b)

doc/install/get-packages: point to current stable release

Point to pacific for downloading the cephadm script

Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Sandro Bonazzola <sbonazzo@redhat.com>
(cherry picked from commit 136dbdef663a5d1c658dad52a794941ab6e6eb07)

doc: Add kcli utilization for development environments

How to use kcli to create ceph clusters (cephadm) and made easy code changes in
manager modules.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 3b82624e0cf0babec6868c1a999c5740fe04302f)

Merge pull request #42027 from rhcs-dashboard/wip-51148-pacific

pacific: mgr/dashboard: Include Network address and labels on Host Creation form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

logrotate: include cephfs-mirror daemon

Fixes: http://tracker.ceph.com/issues/51318
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 957bae255c9f6fba8a4ccfcb885c2687784444aa)

cephfs-mirror: reopen logs on SIGHUP

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 68ad416766e2c73109dbf44f179d1b85b9d61994)