14.2.18

Merge pull request #40116 from votdev/issue_49802

nautilus: mgr/dashboard: Python2 Cookie module import fails on Python3

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #40095 from jdurgin/wip-typing-nautilus

nautilus: mgr/volumes: don't require typing

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Python2 Cookie module import fails on Python3

This fix only affects Nautilus because all successor versions are based on Python3 only.

Fixes: https://tracker.ceph.com/issues/49802
Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/volumes: don't require typing

This isn't available for python2 on centos 7. This is only needed
for nautilus because that's the last release using python 2 and centos 7.

Fixes: https://tracker.ceph.com/issues/49762
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40064 from idryomov/wip-krbd-xfstests-fixes-nautilus

nautilus: qa/suites/krbd: address recent issues caused by newer kernels

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39926 from tchaikov/nautilus-pr-triage

nautilus: .github: add workflow for adding labels and milestone

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

14.2.17

qa/suites/krbd: bump scratch image size to 15G

Allow generic/038 and generic/048 to run.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 37d56e13544fa06b88196b1a4d6edd1b3425325e)

qa/suites/krbd: exclude ext4/002

ext4/002 exercises obsolete EXT4_EOFBLOCKS_FL feature that was removed
in kernel 5.7 and therefore always fails.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d2bdf0ac43a35f755ea70103f497625d882a4592)

qa/tasks/rbd: fix xfstests_branch/url example

- don't mention xfstests_branch together with xfstests_url as
  xfstests_branch gets superseded by xfstests_url
- make xfstests_url example more clear

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ec36d501ed5f875d26d1362ec097187697170a4d)

Merge pull request #39960 from neha-ojha/wip-49672-nautilus

nautilus: qa/tasks/ceph_manager.py: don't use log-early in raw_cluster_cmd

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks/ceph_manager.py: don't use log-early in raw_cluster_cmd

This option is not present in older versions due to which upgrade tests
using the nautilus version of the task will fail.

Caused by https://github.com/ceph/ceph/pull/37906

Fixes: https://tracker.ceph.com/issues/49672
Signed-off-by: Neha Ojha <nojha@redhat.com>

.github/workflows: use a released sha1

otherwise the action cannot be found

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ae8ea10f415ec170be4a7be48567dbac83221dc0)

.github/workflows: rename labeler.yml to pr-triage.yml

to reflect the change in the actions in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 453cd5fd602c54809dc091f0c3a498f903366c8f)

.github/workflows: use @{sha1} for actions

more secure this way. see also https://julienrenaux.fr/2019/12/20/github-actions-security-risk/

point the sha1 to

* labeler@v3
* milestone@main HEAD

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 79e8038046092053b0c0b120e0d7ca07a33a1c00)

.github/workflows: move milestone.yml into labeler.yml

no need to have two actions triggered by the same event.

we could backport this labeler.yml to LTS branches.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 98b90189791d3f4d2524fd829edb616d0ffd94f7)

.github: correct the regex in mileston workflow

also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c5f6d15e36fdd019196a1520525382f73f276f14)

.github: add workflow for adding milestone

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1657a44750442ff2582d601fb028aa89e3a95999)

github/labeler: disable sync-labels

Yaml syntax cleaned too.

Fixes: https://github.com/ceph/ceph/pull/38107#issuecomment-729300615
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 59702b6198c59b84f1695e37256ae351b331b604)

github: autolabel PRs

... starting with dashboard ones.

Requires https://github.com/marketplace/actions/labeler

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 298bc67f7b8d1c64520b543f4749b29f7be67379)

Conflicts:
.github/CODEOWNERS: drop change in .github/CODEOWNERS
          as we don't have this file in nautilus.

Merge pull request #39837 from rhcs-dashboard/wip-49596-nautilus

nautilus: mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Fixes: https://tracker.ceph.com/issues/49574
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 4b96bb51e8f133badd7bc651bcb4dcd755b43d75)

Conflicts:
       src/pybind/mgr/dashboard/controllers/auth.py
         - Resolved branch divergence conflicts.
       src/pybind/mgr/dashboard/services/auth.py
         - Resolved branch divergence  conflicts.
       src/pybind/mgr/dashboard/requirements.txt
         - Removed specific version (already done in master branch).

Merge pull request #39720 from ifed01/wip-ifed-fix-no-cache-trim-nau

nautilus: os/bluestore: go beyond pinned onodes while trimming the cache.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39747 from pcuzner/wip-49540-nautilus

nautilus: mgr/prometheus: Add SLOW_OPS healthcheck as a metric

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39756 from ifed01/wip-ifed-zero-read-retry-nau

nautilus: os/bluestore: Add option to check BlueFS reads

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39646 from rhcs-dashboard/wip-49389-nautilus

nautilus: mgr/dashboard: fix tooltip for Provisioned/Total Provisioned fields

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39526 from rhcs-dashboard/wip-49327-nautilus

nautilus: mgr/dashboard: avoid using document.write()

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39342 from smithfarm/wip-49202-nautilus

nautilus: common: skip interfaces starting with "lo" in find_ipv{4,6}_in_subnet()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39340 from smithfarm/wip-48897-nautilus

nautilus: mgr: fix deadlock in ActivePyModules::get_osdmap()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39339 from smithfarm/wip-48558-nautilus

nautilus: mgr/restful: fix TypeError occurring in _gather_osds()

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #39338 from smithfarm/wip-48575-nautilus

nautilus: mgr/crash: Serialize command handling

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #39255 from ifed01/wip-ifed-fsck-spanning-nau

nautilus: os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39126 from smithfarm/wip-48987-nautilus

nautilus: core: osd: An empty bucket or OSD is not an error

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #39543 from aaSharma14/wip-48588-nautilus

nautilus: mgr/dashboard: RGW User Form is validating disabled fields

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39470 from jan--f/wip-49290-nautilus

nautilus: ceph-volume: Update batch.py

os/bluestore: go beyond pinned onodes while trimming the cache.

One might face lack of cache trimming when there is a bunch of pinned entries on the top of Onode's cache LRU list. If these pinned entries stay in the state for a long time cache might start using too much memory causing OSD to go out of osd-memory-target limit. Pinned state tend to happen to osdmap onodes.
The proposed patch preserves last trim position in the LRU list (if it pointed to a pinned entry) and proceeds trimming from that position if it wasn't invalidated. LRU nature of the list enables to do that safely since no new entries appear above the previously present entry while it's not touched.

Fixes: https://tracker.ceph.com/issues/48729
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #39706 from batrick/i48286

nautilus: rados/upgrade/nautilus-x-singleton fails due to cluster [WRN] evicting unresponsive client

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #39276 from smithfarm/wip-49130-nautilus

nautilus: rgw: fix trailing null in object names of multipart reuploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39661 from pponnuvel/wip-49440-nautilus

nautilus: rgw: radosgw-admin: clarify error when email address already in use

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39651 from smithfarm/wip-49055-nautilus

nautilus: common/mempool: Improve mempool shard selection

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #39698 from ifed01/wip-ifed-fix-huge-bluefs-nau

nautilus: os/bluestore: fix huge read/writes in BlueFS

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #39650 from smithfarm/wip-49182-nautilus

nautilus: test/rbd-mirror: fix broken ceph_test_rbd_mirror_random_write

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #39324 from rzarzynski/wip-librados-nul-objname-nautilus

nautilus: librados, tests: allow to list objects with the NUL character in names

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #39708 from smithfarm/wip-48282-nautilus

nautilus: osd: fix bluestore bitmap allocator calculate wrong last_pos with hint

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39509 from k0ste/wip-49314-nautilus

nautilus: mgr/dashboard: prometheus alerting: add some leeway for package drops and errors

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #39329 from rzarzynski/wip-tool-nul-in-rados-nautilus

nautilus: tools/rados: add support for binary object names in the rados CLI

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

os/bluestore: Add option to check BlueFS reads

Add option "bluefs_check_for_zeros" to check if there are any zero-filled page.
If so, reread data. It is known that sometimes BlueStore gets such pages.
See "bluestore_retry_disk_reads".

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 7e495c57fe0ec8d8bdb1dbff90b177e656a22cbe)

 Conflicts:
 (lack of fresh stuff in bufferlist) src/os/bluestore/BlueFS.cc

mgr: add doc to describe relationship to prometheus

The healthcheck for slow_ops is used by the mgr/prometheus
module so it's important the alert generated matches the
format expected in mgr/prometheus.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit fc1905b4a74aedc6bdbc928b6d6e514183bbf090)

Conflicts:
src/mgr/DaemonHealthMetricCollector.cc

- docstring added to describe the link to mgr/prometheus conflicted with the
  const fmt definition for the message. resolved by adding doc under the const
  definition.

mgr/prometheus: Add healthcheck metric for SLOW_OPS

SLOW_OPS is triggered by op tracker, and generates a health
alert but healthchecks do not create metrics for prometheus to
use as alert triggers. This change adds SLOW_OPS metric, and
provides a simple means to extend to other relevant health
checks in the future

If the extract of the value from the health check message fails
we log an error and remove the metric from the metric set. In
addition the metric description has changed to better reflect
the scenarios where SLOW_OPS can be triggered.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 2010432b5045c8f1dd0c052def497e590a1bf2fe)

Conflicts:
src/pybind/mgr/prometheus/module.py
- Nautilus doesn't have the profile_method decorator, so needed to be removed

Merge pull request #39441 from rhcs-dashboard/49278-py2-unicode-password

nautilus: mgr/dashboard: python 2: error when setting non-ASCII password

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39129 from smithfarm/wip-48376-nautilus

nautilus: cephfs: client: check rdonly file handle on truncate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37906 from ideepika/backport-a7f18e4

nautilus: qa/tasks/{ceph,ceph_manager}: drop py2 support

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

osd: fix bluestore bitmap allocator calculated wrong last_pos with hint

Fixes: https://tracker.ceph.com/issues/48214
Signed-off-by: Xue Yantao <jhonxue@tencent.com>
(cherry picked from commit ad3c7cdf6fabbb4a8b542b793d632cae87ee3640)

common/mempool: include standard thread library

Attempt to address FTBFS:

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_mempool.cc:399:11: error: request for member 'clear' in 'workers', which is of non-class type 'int'
  399 |   workers.clear();
      |           ^~~~~

seen in nautilus.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa: ignore evicted client warnings for nautilus upgrades

Nautilus monitors do not note the client sessions in the mgrmap. So when
we upgrade the warnings are unavoidable.

Fixes: https://tracker.ceph.com/issues/47689
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2163c60916bb1e5249dcff3e33822ddb1ead4653)

Conflicts:
      qa/suites/upgrade/mimic-x/parallel/1-ceph-install/mimic.yaml
      qa/suites/upgrade/nautilus-x-singleton/1-install/nautilus.yaml
      qa/suites/upgrade/nautilus-x/stress-split/1-ceph-install/nautilus.yaml

mgr: log reason for respawn

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b835b07bc3c2936169dc3e6a212a2e29885246f7)

Merge pull request #39001 from Vicente-Cheng/wip-48520-nautilus

nautilus: client: add ceph.{cluster_fsid/client_id} vxattrs suppport

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

common/mempool: Add test for mempool in btree_map

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit b426e52d0425f3ececf48c7936d3d6d0eeb69ebf)

Conflicts:
src/test/test_mempool.cc
- tests went into nautilus in a different order

Merge pull request #39544 from aaSharma14/wip-48733-nautilus

nautilus: mgr/dashboard: minimize console log traces of Ceph backend API tests

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #39518 from aaSharma14/wip-49323-nautilus

nautilus: mgr/dashboard: fix MTU Mismatch alert

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38885 from yanghonggang/wip-48857-nautilus

nautilus: rgw: multisite: Verify if the synced object is identical to source

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common/mempool: copies bufferlist assignment to mempool through rebuild

Fixes: https://tracker.ceph.com/issues/46027
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit b3a6d18ea64e0f70e0d3371cbfcd21280677000e)

Conflicts:
src/test/test_mempool.cc
- nautilus version of this file has less tests than master

nautilus: mgr/dashboard: python 2: error when setting user's non-ASCII password

Fixes: https://tracker.ceph.com/issues/49278
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #39658 from kotreshhr/ceph-vol-client-update-fix-volumes-key-nautilus

nautilus: pybind/ceph_volume_client: Update the 'volumes' key to 'subvolumes' in auth-metadata file

Reviewed-by: Ramana Raja <rraja@redhat.com>

os/bluestore: fix huge (>4GB) bluefs reads

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fe558c4dd12b25b2d523987f22fde5b5898cfb83)

 Conflicts:
 (still present buf param in BlueFS::read, lack of )https://github.com/ceph/ceph/pull/34421)
src/os/bluestore/BlueFS.cc
src/os/bluestore/BlueFS.h
src/os/bluestore/BlueRocksEnv.cc
src/test/objectstore/test_bluefs.cc

os/bluestore: fix huge(>4GB) writes from RocksDB to BlueFS.

Fixes: https://tracker.ceph.com/issues/49168
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 5f94883ec8d64c02b2bb499caad8eaf91dd715f7)

 Conflicts:
 (lack of bufferlist refactor from https://github.com/ceph/ceph/pull/36754)
 (lack of single allocator support from https://github.com/ceph/ceph/pull/30838)
src/os/bluestore/BlueFS.h
src/test/objectstore/test_bluefs.cc

test/unittest_bluefs: always remove temp bdev file

we leave files in build directory if the test fails. better off
removing them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a38122fdf77378d67eb97a5a2c55805181d921c2)

os/bluestore/BlueRocksEnv: Avoid flushing too much data at once.

Although, in _flush func we already check length. If length of dirty
is less then bluefs_min_flush_size, we will skip this flush.
But in fact, we found rocksdb can call many times Append() and then
call Flush(). This make flush_data is much larger than
bluefs_min_flush_size.

From my test, w/o this patch, it can reduce 99.99% latency(from
145.753ms to 20.474ms) for 4k randwrite with bluefs_buffered_io=true.

Because Bluefs::flush acquire lock. So we add new api try_flush
to avoid lock contention.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 3a1d4ff6dcdcd897d31a85ef82c726599624b66d)

mgr/volumes: Bump up AuthMetadataManager's version

With ceph_volume_client and mgr-volumes co-existing
for sometime, the version of both needs to be same.
The ceph_volume_client version <=5 can't decode
'subvolumes' key in auth-metadata file. Hence to
handle version in-compatibility, the version of
ceph_volume_client is bumped up to 6 and the same
needs to be done in mgr-volume's AuthMetadataManager

Fixes: https://tracker.ceph.com/issues/49374
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 818c7781ff0467c6521bf2b3002b094fb8a71257)

pybind/ceph_volume_client: Bump up the version and compat_version to 6

With 'volumes' key updated to 'subvolumes', the version of
ceph_volume_client <= 5 can't decode auth-metadata file. Hence
bumping up ceph_volume_client version and compat_version to 6.

Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit ce55a0bcdc5db139fc8f39665c372f1b7cc274a7)

pybind/ceph_volume_client: Fix auth-metadata file recovery

Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 99cdfdad59175a19bc06859e69301ca0478a2db4)

pybind/ceph_volume_client: Update the 'volumes' key to 'subvolumes' in auth metadata file

The older auth metadata files before nautilus release stores
the authorized subvolumes using the 'volumes' key. As the
notion of 'subvolumes' brought in by mgr/volumes, it makes
sense to use 'subvolumes' key. This patch would be tranparently
update 'volumes' key to 'subvolumes' and newer auth metadata
files would store them with 'subvolumes' key.

Also fails the deauthorize if the auth-id doesn't exist.

Fixes: https://tracker.ceph.com/issues/49294
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit dee03c8d5c0b86cf51865090bec203419a3008a9)

pybind/ceph_volume_client: logger.warn() -> logger.warning()

logger.warn() is the same as logger.warning(), but was removed in py3.3

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fe4c7fd7471bf724c4a1f027580d9bae3ddc24eb)

tools/rados: assign to optional<> without deref'ing it

this change addresses a regression introduced by
d333b35aa10bf03a8bc047994d5cf3fed019b49a

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 41d68594ae6df343782a1c382ac0bddc8cb7cb69)

rgw/radosgw-admin clarify error when email address already in use

The error message if you try and create an S3 user with an email
address that is already associated with another S3 account is very
confusing; this patch makes it much clearer

To reproduce:

radosgw-admin user create --uid=foo --display-name="Foo test" --email=bar@domain.invalid
radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to parse parameters, user id mismatch, operation id: foo does not match: test

With this patch:

radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to create user test because user id foo already exists with email bar@domain.invalid

Fixes: https://tracker.ceph.com/issues/49137
Fixes: https://tracker.ceph.com/issues/19411
Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 05318d6f71e45a42a46518a0ef17047dfab83990)

common: move gen_rand_alphanumeric() helpers into common

and adds overloads that return std::string directly

the motivation is to remove cls_otp's dependency on rgw_common.h

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit bfa76731aa338ca2c8a9e6dc52a19ae71aeb0bbf)

Conflicts:
src/common/CMakeLists.txt
- no rabin.cc in nautilus

test: add unit tests for gen_rand_alphanumeric helpers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fb8975e0d4ce618c0b61aff3578e71706d23ae22)

rgw: fix trailing null in object names of multipart reuploads

the overload being called had a comment saying:
/* size should be the required string size + 1 */

this calls a different overload that returns std::string and handles
this null termination correctly

Fixes: https://tracker.ceph.com/issues/48874
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2ede30782b00fe28ac714047f55b11b76ae1593f)

common/mempool: Modify shard selection function

Modify shard selection function, so no longer all threads use the same shard 0.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit fccbdc0905e3868666fbb10803bac6b73f687cb1)

common/mempool: Add test for mempool shards

Add test that checks quality of mempool sharing.
It refuses to accept cases when variance is too large.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit e4e07de8f36d0d44b341d5784a8bf56192201b95)

Conflicts:
src/test/test_mempool.cc
- nautilus version does not have as many tests as master version

test/rbd-mirror: fix broken ceph_test_rbd_mirror_random_write

It appears that commit 6eb8f30a238 broke the test utility and
its failure was masked by the test case that expected a failure
due to a timeout force-killing the app.

Fixes: https://tracker.ceph.com/issues/49117
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8643b046fb4d5b05b4c75b83f16cd8ccc6a8b0a0)

Conflicts:
qa/workunits/rbd/rbd_mirror_helpers.sh
- no show_diff function in nautilus

mgr/dashboard: fix tooltip for Provisioned/Total Provisioned fields

Fixes: https://tracker.ceph.com/issues/46619
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 2dd341d63b7dd7d6b8767b93151c0c1f243be6b6)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.html
      - Adopting the master branch changes.

Merge pull request #39626 from rhcs-dashboard/wip-49420-nautilus

nautilus: mgr/dashboard: set security headers

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

qa/tasks/ceph_manager: use s/ByteIO/StringIO in stdout for ceph-objectstore-tool

wrt master, we have moved to using run_ceph_objectstore_tool which uses
StringIO for stdout and stderr, to make the changes compatible with
nautilus, replacing use of ByteIO with StringIO.

see also: d8d44ed

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #39584 from liewegas/pr-39561-nautilus

nautilus: qa: use normal build for valgrind

qa/valgrind.supp: ignore valgrind leak

<error>
  <unique>0x12ee80e</unique>
  <tid>1</tid>
  <threadname>ceph-mon</threadname>
  <kind>Leak_StillReachable</kind>
  <xwhat>
    <text>8 bytes in 1 blocks are still reachable in loss record 2 of 31</text>
    <leakedbytes>8</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0x4C2AC38</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new[](unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.15.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>433</line>
    </frame>
    <frame>
      <ip>0xEB02D40</ip>
      <obj>/usr/lib64/libtcmalloc.so.4.4.5</obj>
    </frame>
    <frame>
      <ip>0x400F9B2</ip>
      <obj>/usr/lib64/ld-2.17.so</obj>
      <fn>_dl_init</fn>
    </frame>
    <frame>
      <ip>0x4001179</ip>
      <obj>/usr/lib64/ld-2.17.so</obj>
    </frame>
    <frame>
      <ip>0x5</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BBE</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BC7</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BCA</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BD4</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BD9</ip>
    </frame>
    <frame>
      <ip>0x1FFF000BDC</ip>
    </frame>
  </stack>
</error>

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/dashboard: set security headers

Fixes: https://tracker.ceph.com/issues/49243
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 87bb3777877580f70018f4880488e264decc16e4)

 Conflicts:
qa/tasks/mgr/dashboard/test_requests.py
src/pybind/mgr/dashboard/services/auth.py
      - Adopting the master branch changes.

rgw multisite: fix single-part-MPU object etag misidentify problem

The single-part-MPU object is treated as a Atomic object.
Because its part_size is 0.

As MPU object's start_part_num is not 0 which can be used to
distinguish MPU object and Atomic object.

Fixes: https://tracker.ceph.com/issues/49357
Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
(cherry picked from commit 63f989ca2bdc43142713b5da1eee2668ba238716)

qa/valgrind.supp: suppress free[] warnings

This is apparently not going to get fixed any time soon.

https://github.com/gperftools/gperftools/issues/792

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 28fe149ea276693b47cc4d600780971f723672e4)

qa/suites: do not use notcmalloc flavor

teuthology now knows how to run valgrind against a tcmalloc binary

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dc64ccf063f1ddf843fae4d2621ec25187f56135)

# Conflicts:
# qa/suites/rados/valgrind-leaks/1-start.yaml
# qa/suites/rgw/multisite/valgrind.yaml
# qa/suites/rgw/sts/tasks/0-install.yaml
# qa/suites/rgw/verify/validater/valgrind.yaml
# qa/suites/rgw/website/overrides.yaml

mgr/dashboard:minimize console log traces of Dashboard backend API tests

Changed log levels in some files to reduce console log traces

Fixes: https://tracker.ceph.com/issues/45822
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit f5dd0195cd45b6e00e43ef97e209e565e6452777)

mgr/dashboard: RGW User Form is validating disabled fields

When you try to submit a non valid form, angular is checking all the fields of the form, resulting in some wrong validations.

Fixes:https://tracker.ceph.com/issues/47727
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 071d1f317de547bf8fae68a0700d429900d3a741)

tools/rados: replace omap_key_valid with std::optional.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit eb87a0646a3635a7a35b6326e6d26e4b0b5b9d37)

tools/rados: replace omap_key_pretty with prettify().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit cca92228f279d1334207b5e1f4bcfa225dca7f11)

tools/rados: add support for binary object names.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d333b35aa10bf03a8bc047994d5cf3fed019b49a)

Conflicts:
src/tools/rados/rados.cc
  * no input-file option,
  * no commands for cache tiering,
  * older API for omap retrival.

Merge pull request #39438 from rhcs-dashboard/wip-49271-nautilus

nautilus: mgr/dashboard: delete EOF when reading passwords from file

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #39197 from ybwang0211/CrushWrapper

nautilus: crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39160 from smithfarm/wip-48495-nautilus

nautilus: mon: paxos: Delete logger in destructor

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

mgr/dashboard: avoid using document.write()

Fixes: https://tracker.ceph.com/issues/49220
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 36efe519b90b7508fb830f2c250e8b6956eaaf05)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/app.module.ts
     - Adopt the master branch changes.