]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoqa: retry test_rbd_wnbd.py fs operations 49618/head
Lucian Petrut [Tue, 3 Jan 2023 09:56:30 +0000 (11:56 +0200)]
qa: retry test_rbd_wnbd.py fs operations

Certain FS related operations can fail, especially under load
(e.g. initializing partitions, volume formatting, etc).

For this reason, we're going to introduce some retries.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoqa: add some retries to test_rbd_wnbd.py 49532/head
Lucian Petrut [Thu, 22 Dec 2022 13:26:36 +0000 (15:26 +0200)]
qa: add some retries to test_rbd_wnbd.py

The following operations may fail right after a block device
is attached:

* retrieving the disk number (can return -1)
* opening the disk
* setting the disk online or writable

For this reason, we'll need to add some retries. For convenience,
we're moving the existing retry logic to a separate decorator.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoqa: fix test_rbd_wnbd.py, properly retrieving the drive letter
Lucian Petrut [Wed, 21 Dec 2022 13:58:07 +0000 (15:58 +0200)]
qa: fix test_rbd_wnbd.py, properly retrieving the drive letter

Instead of trying to use the first partiton which may be reserved
by Windows, we'll fetch the first non-empty drive letter from
the disk that we've just mounted.

While at it, we're ensuring that the drive letter is actually a
letter and not a null character, which the Powershell command
returns in case of empty drive letters.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoMerge pull request #49403 from tchaikov/cmake-boost
Kefu Chai [Wed, 21 Dec 2022 13:18:04 +0000 (21:18 +0800)]
Merge pull request #49403 from tchaikov/cmake-boost

cmake: pass cflags using compileflags in BuildBoost.cmake

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #49305 from petrutlucian94/dokan_subsys
Lucian Petrut [Wed, 21 Dec 2022 11:09:55 +0000 (13:09 +0200)]
Merge pull request #49305 from petrutlucian94/dokan_subsys

dokan: use the right logging subsystem

2 years agoMerge pull request #48929 from petrutlucian94/rbd_py_test
Ilya Dryomov [Wed, 21 Dec 2022 10:42:57 +0000 (11:42 +0100)]
Merge pull request #48929 from petrutlucian94/rbd_py_test

qa: fix test_rbd_wnbd.py

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #49497 from BryceCao/wip-fix-mfa.rst-spelling-mistake
Anthony D'Atri [Wed, 21 Dec 2022 03:28:49 +0000 (22:28 -0500)]
Merge pull request #49497 from BryceCao/wip-fix-mfa.rst-spelling-mistake

doc/radosgw: fix spelling mistake in mfa.rst

2 years agodoc/radosgw: fix spelling mistake in mfa.rst 49497/head
cao.leilc [Mon, 19 Dec 2022 02:52:19 +0000 (10:52 +0800)]
doc/radosgw: fix spelling mistake in mfa.rst

Signed-off-by: caolei <cao.leilc@inspur.com>
2 years agoMerge pull request #49402 from BryceCao/wip-fix-usage-trim-cant-be-control-by-remove-all
Casey Bodley [Tue, 20 Dec 2022 19:32:29 +0000 (14:32 -0500)]
Merge pull request #49402 from BryceCao/wip-fix-usage-trim-cant-be-control-by-remove-all

rgw: fix problem that usage is trimmed when the remove-all parameter …

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49390 from cbodley/wip-qa-rgw-inline
Casey Bodley [Tue, 20 Dec 2022 19:32:16 +0000 (14:32 -0500)]
Merge pull request #49390 from cbodley/wip-qa-rgw-inline

qa/rgw: rgw/verify suite randomizes --placement-inline-data

Reviewed-by: Cory Snyder <csnyder@iland.com>
2 years agoMerge pull request #49387 from cfsnyder/wip-cfsnyder-58243
Casey Bodley [Tue, 20 Dec 2022 19:31:54 +0000 (14:31 -0500)]
Merge pull request #49387 from cfsnyder/wip-cfsnyder-58243

rgw: fix dereference of empty std::optional in RGWDeleteMultiObj::execute()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49362 from cbodley/wip-58231
Casey Bodley [Tue, 20 Dec 2022 19:31:37 +0000 (14:31 -0500)]
Merge pull request #49362 from cbodley/wip-58231

rgw/sal: get_notification() takes optional_yield

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #49324 from BryceCao/wip-add-num-limit-for-user-policies
Casey Bodley [Tue, 20 Dec 2022 19:30:59 +0000 (14:30 -0500)]
Merge pull request #49324 from BryceCao/wip-add-num-limit-for-user-policies

rgw : add num limit for IAM user policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49298 from BryceCao/wip-fix-the-inheriting-class-member-is-passed...
Casey Bodley [Tue, 20 Dec 2022 19:30:20 +0000 (14:30 -0500)]
Merge pull request #49298 from BryceCao/wip-fix-the-inheriting-class-member-is-passed-to-base-ctor

rgw : fix the inheriting class member is passed to base ctor

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #49194 from joke-lee/list-delimiter-encode
Casey Bodley [Tue, 20 Dec 2022 19:29:58 +0000 (14:29 -0500)]
Merge pull request #49194 from joke-lee/list-delimiter-encode

rgw: list with Delimiter should url encode when encoding-type=url

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49426 from BryceCao/wip-fix-ListOpenIDConnectProviders-XML-format
Casey Bodley [Tue, 20 Dec 2022 19:29:36 +0000 (14:29 -0500)]
Merge pull request #49426 from BryceCao/wip-fix-ListOpenIDConnectProviders-XML-format

rgw: fix ListOpenIDConnectProviders XML format

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge PR #49016 into main
Patrick Donnelly [Tue, 20 Dec 2022 18:19:17 +0000 (13:19 -0500)]
Merge PR #49016 into main

* refs/pull/49016/head:
doc/dev: remove release notes for major release branch
doc: hard-code latest release link
doc: remove rtd main checkout

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge PR #48743 into main
Venky Shankar [Tue, 20 Dec 2022 14:22:58 +0000 (19:52 +0530)]
Merge PR #48743 into main

* refs/pull/48743/head:
mds: do not acquire xlock in xlockdone state

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #48843 from rhcs-dashboard/expose_slow_ops
Pere Diaz Bou [Tue, 20 Dec 2022 11:25:32 +0000 (12:25 +0100)]
Merge pull request #48843 from rhcs-dashboard/expose_slow_ops

mgr/prometheus: expose daemon health metrics

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #47276 from rhcs-dashboard/mirror-pool-health
Pedro Gonzalez Gomez [Tue, 20 Dec 2022 09:49:31 +0000 (10:49 +0100)]
Merge pull request #47276 from rhcs-dashboard/mirror-pool-health

mgr/dashboard: display real health in rbd mirroring pools

Reviewed-by: Pegonzal <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agomgr/prometheus: expose daemon health metrics 48843/head
Pere Diaz Bou [Fri, 11 Nov 2022 09:43:01 +0000 (10:43 +0100)]
mgr/prometheus: expose daemon health metrics

Until now daemon health metrics were stored without being used. One of
the most helpful metrics there is SLOW_OPS with respect to OSDs and MONs
which this commit tries to expose to bring fine grained metrics to find
troublesome OSDs instead of having a lone healthcheck of slow ops in the
whole cluster.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #49496 from cyx1231st/wip-seastore-fix-denc
Yingxin [Tue, 20 Dec 2022 05:27:07 +0000 (13:27 +0800)]
Merge pull request #49496 from cyx1231st/wip-seastore-fix-denc

crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #49509 from rhcs-dashboard/fix-prometheus-module-blank-line
Kefu Chai [Tue, 20 Dec 2022 04:03:34 +0000 (12:03 +0800)]
Merge pull request #49509 from rhcs-dashboard/fix-prometheus-module-blank-line

mgr/prometheus: Fix make check flake8 blank line missing

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #49514 from zdover23/wip-doc-2022-12-20-glossary-cephx-config...
colemitchell [Tue, 20 Dec 2022 00:41:17 +0000 (19:41 -0500)]
Merge pull request #49514 from zdover23/wip-doc-2022-12-20-glossary-cephx-config-ref-link

doc/glossary: link to CephX Config ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #49513 from zdover23/wip-doc-2022-12-20-rados-auth-config-ref...
colemitchell [Mon, 19 Dec 2022 18:18:59 +0000 (13:18 -0500)]
Merge pull request #49513 from zdover23/wip-doc-2022-12-20-rados-auth-config-ref-prompts

doc/rados: add prompts to auth-config-ref.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/glossary: link to CephX Config ref 49514/head
Zac Dover [Mon, 19 Dec 2022 18:16:19 +0000 (04:16 +1000)]
doc/glossary: link to CephX Config ref

Link to the CephX Configuration Reference in the "CephX" entry in the
glossary.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agodoc/rados: add prompts to auth-config-ref.rst 49513/head
Zac Dover [Mon, 19 Dec 2022 18:00:59 +0000 (04:00 +1000)]
doc/rados: add prompts to auth-config-ref.rst

Add unselectable prompts to doc/rados/configuration/auth-config-ref.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49510 from zdover23/wip-doc-2022-12-20-rados-config-msgr2-prompts
colemitchell [Mon, 19 Dec 2022 17:50:46 +0000 (12:50 -0500)]
Merge pull request #49510 from zdover23/wip-doc-2022-12-20-rados-config-msgr2-prompts

doc/rados: add prompts to msgr2.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/rados: add prompts to msgr2.rst 49510/head
Zac Dover [Mon, 19 Dec 2022 17:41:22 +0000 (03:41 +1000)]
doc/rados: add prompts to msgr2.rst

Add unselectable prompts to doc/rados/configuration/msgr2.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agomgr/prometheus: add blank like previously wrongly deleted that was causing issues... 49509/head
Pedro Gonzalez Gomez [Mon, 19 Dec 2022 15:07:31 +0000 (16:07 +0100)]
mgr/prometheus: add blank like previously wrongly deleted that was causing issues on makecheck due to flake8

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2 years agoMerge PR #48473 into main
Venky Shankar [Mon, 19 Dec 2022 10:47:14 +0000 (16:17 +0530)]
Merge PR #48473 into main

* refs/pull/48473/head:
qa: fix _run_mount_cmd() and _get_mount_cmd() args in vstart_runner

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2 years agoMerge pull request #49490 from tchaikov/wip-pybind-mgr-tox-fixes
Kefu Chai [Mon, 19 Dec 2022 10:14:13 +0000 (18:14 +0800)]
Merge pull request #49490 from tchaikov/wip-pybind-mgr-tox-fixes

pybind/mgr: tox and test fixes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #49049 from rhcs-dashboard/add-osd-blocklist-metric
Aashish Sharma [Mon, 19 Dec 2022 08:07:12 +0000 (13:37 +0530)]
Merge pull request #49049 from rhcs-dashboard/add-osd-blocklist-metric

mgr/dashboard: Add metric relative to osd blocklist

2 years agoMerge pull request #49292 from rhcs-dashboard/rbd-mirror-improvements
Aashish Sharma [Mon, 19 Dec 2022 06:49:58 +0000 (12:19 +0530)]
Merge pull request #49292 from rhcs-dashboard/rbd-mirror-improvements

mgr/dashboard: rbd-mirror improvements

2 years agoMerge pull request #49203 from ktdreyer/doc-simple-cephfs-create
Anthony D'Atri [Mon, 19 Dec 2022 06:46:06 +0000 (01:46 -0500)]
Merge pull request #49203 from ktdreyer/doc-simple-cephfs-create

doc/cephfs: simplify ceph fs volume create command

2 years agocrimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t 49496/head
Yingxin Cheng [Mon, 19 Dec 2022 02:47:46 +0000 (10:47 +0800)]
crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t

* device_spec_t should not include denc and be packed at the same time
* the underlying type of enum:uint8_t should support denc_traits already

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2 years agoMerge pull request #49489 from tchaikov/wip-cmake-with-fmt-version
Kefu Chai [Mon, 19 Dec 2022 00:47:29 +0000 (08:47 +0800)]
Merge pull request #49489 from tchaikov/wip-cmake-with-fmt-version

cmake: set WITH_FMT_VERSION as a STRING option

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #49491 from zdover23/wip-doc-2022-12-18-rados-config-ceph-conf...
colemitchell [Sun, 18 Dec 2022 18:09:54 +0000 (13:09 -0500)]
Merge pull request #49491 from zdover23/wip-doc-2022-12-18-rados-config-ceph-conf-prompts

doc/rados: add prompts to ceph-conf.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: add prompts to ceph-conf.rst 49491/head
Zac Dover [Sun, 18 Dec 2022 15:29:33 +0000 (01:29 +1000)]
doc/rados: add prompts to ceph-conf.rst

Add unselectable prompts to doc/rados/ceph-conf.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agopybind/mgr/tox.ini: add commas in "modules" variable 49490/head
Kefu Chai [Sun, 18 Dec 2022 12:18:44 +0000 (20:18 +0800)]
pybind/mgr/tox.ini: add commas in "modules" variable

since tox v4.0.13, it parses the variables differently, so the newlines
in a variable are passed right to the command referencing it. so we now
have failure like:
```
flake8: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> flake8 --config=tox.ini alerts
flake8: commands[1] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: FAIL ✖ in 3.33 seconds
```
so we have to add comma as line continuation separator to address
this problem.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agopybind/mgr: s/setup(self)/setup_method(self)/
Kefu Chai [Sun, 18 Dec 2022 12:16:02 +0000 (20:16 +0800)]
pybind/mgr: s/setup(self)/setup_method(self)/

avoid pytest warnings like:

4: pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk
4:   /home/kefu/dev/ceph/src/pybind/mgr/.tox/py3/lib/python3.10/site-packages/_pytest/fixtures.py:900: PytestRemovedIn8Warning: Support for nose tests is deprecated and will be removed in a future release.
4:   pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk is using nose-specific method: `setup(self)`
4:   To remove this warning, rename it to `setup_method(self)`
4:   See docs: https://docs.pytest.org/en/stable/deprecations.html#support-for-tests-written-for-nose
4:     fixture_result = next(generator)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agopybind/mgr/prometheus: avoid using distutils
Kefu Chai [Sun, 18 Dec 2022 12:15:06 +0000 (20:15 +0800)]
pybind/mgr/prometheus: avoid using distutils

to silence warnings like:

4: prometheus/module.py:35
4:   /var/ssd/ceph/src/pybind/mgr/prometheus/module.py:35: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
4:     v = StrictVersion(cherrypy.__version__)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agocmake: set WITH_FMT_VERSION as a STRING option 49489/head
Kefu Chai [Sun, 18 Dec 2022 02:47:42 +0000 (10:47 +0800)]
cmake: set WITH_FMT_VERSION as a STRING option

option() defines a boolean, so its default value cannot be populated
to find_package() as expected. so we need to set it as a cached STRING
variable.

this addresses the regression introduced by 051ec194df16927187cbe4b26dacd06c31fa186a

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #49430 from tchaikov/wip-fmtlib-v9
Kefu Chai [Sat, 17 Dec 2022 07:08:29 +0000 (15:08 +0800)]
Merge pull request #49430 from tchaikov/wip-fmtlib-v9

cmake, script: run "make check" with fmtlib v9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49394 from zdover23/wip-doc-2022-12-10-rbd-rbd-snapshot-edit
zdover23 [Fri, 16 Dec 2022 15:03:22 +0000 (01:03 +1000)]
Merge pull request #49394 from zdover23/wip-doc-2022-12-10-rbd-rbd-snapshot-edit

doc/rbd: refine rbd-snapshot.rst

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agodoc/rbd: refine rbd-snapshot.rst 49394/head
Zac Dover [Fri, 9 Dec 2022 17:42:14 +0000 (03:42 +1000)]
doc/rbd: refine rbd-snapshot.rst

Refine the text in rbd-snapshot.rst

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49479 from tchaikov/wip-rgw-fmt-v8
Casey Bodley [Fri, 16 Dec 2022 14:19:12 +0000 (09:19 -0500)]
Merge pull request #49479 from tchaikov/wip-rgw-fmt-v8

rgw: be compatible with fmtlib v8

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoosd/scrubber: mark fmt::formatter<..>::format() as const 49430/head
Kefu Chai [Fri, 16 Dec 2022 12:05:43 +0000 (20:05 +0800)]
osd/scrubber: mark fmt::formatter<..>::format() as const

as it should be const, and more importantly, it'd fail the build with
fmtlib v9, as its caller(s) is marked as `const`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #49173 from rhcs-dashboard/fix-encryption-config
Aashish Sharma [Fri, 16 Dec 2022 11:35:13 +0000 (17:05 +0530)]
Merge pull request #49173 from rhcs-dashboard/fix-encryption-config

mgr/dashboard: fix server side encryption config error

2 years agorgw: be compatible with fmtlib v8 49479/head
Kefu Chai [Fri, 16 Dec 2022 11:14:53 +0000 (19:14 +0800)]
rgw: be compatible with fmtlib v8

before fmtlib v9, fmtlib provides:

template<typename S, typename ...Args, typename Char = enable_if_t<detail::is_string<S>::value, char_t<S>>>
void fmt::print(std::basic_ostream<Char> &os, const S &format_str, Args&&... args)

but in fmtlib v9 and up, it provides:

template<typename ...T>
void fmt::print(std::ostream &os, format_string<T...> fmt, T&&... args)

so we need to use different function signatures for talking to different
fmtlib versions.

please see
https://fmt.dev/8.1.0/api.html#_CPPv4I0Dp0EN3fmt5printEvRNSt13basic_ostreamI4CharEERK1SDpRR4Args,
and
https://fmt.dev/9.0.0/api#_CPPv4IDpEN3fmt5printEvRNSt7ostreamE13format_stringIDp1TEDpRR1T

fore more details.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agomgr/dashboard: Rbd Mirroring improvements 49292/head
Aashish Sharma [Wed, 7 Dec 2022 05:25:24 +0000 (10:55 +0530)]
mgr/dashboard: Rbd Mirroring improvements

1.RBD images in dashboard shows default mirroring as journal
2.snapshot based mirroring schedule Interval got disabled to edit
3.unable to create snapshot of an image using dashboard
4.provide snapshot schedule info in a new column
5.dashboard doesn't allow importing peer bootstrap key to be imported for subsequent pools

Fixes: https://tracker.ceph.com/issues/58297
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 1a37c5e001e0bbc2720e6dfabf6ba04442acabb7)

2 years agoqa: add fs tests to test_rbd_wnbd.py 48929/head
Lucian Petrut [Mon, 12 Dec 2022 10:13:41 +0000 (12:13 +0200)]
qa: add fs tests to test_rbd_wnbd.py

The Windows rbd-wnbd python test performs various IO operations
against raw disks.

However, it can be useful to test overlaying filesystems as well.
For this reason, we're adding the following tests:

* RbdFsTest
* RbdFsFioTest
* RbdFsStampFioTest

To simplify the implementation, those tests reuse the existing
ones along with a mixin class (RbdFsTestMixin).

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoqa: test_rbd_wnbd.py improvements
Lucian Petrut [Wed, 23 Nov 2022 14:50:08 +0000 (16:50 +0200)]
qa: test_rbd_wnbd.py improvements

We'll make the following improvements to the Windows rbd-wnbd
Python test:

* expose fio write validation, defaulting to crc32c
* change the default fio operation to "rw"
* enable the disk and clear the "rw" flag only if required by the
  test and if "--skip-enabling-disk" is not set (useful with custom
  SAN policies). This operation can take a significant amount of
  time under heavy load.
* print fio read and write results separately instead of
  aggregating them, useful when running rw tests

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agodoc: document Windows SAN policy
Lucian Petrut [Wed, 23 Nov 2022 10:53:13 +0000 (12:53 +0200)]
doc: document Windows SAN policy

The Windows SAN policy determines which disks will be mounted
automatically. We're going to document this as well as some
Windows limitations.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoscript/run-make: build with fmtlib 9.0.0
Kefu Chai [Wed, 14 Dec 2022 14:53:21 +0000 (22:53 +0800)]
script/run-make: build with fmtlib 9.0.0

so we can identify changes which breaks the build with fmtlib v9.0
while running "make check" performed by jenkins.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agocmake: add an option named WITH_FMT_VERSION
Kefu Chai [Wed, 14 Dec 2022 14:48:46 +0000 (22:48 +0800)]
cmake: add an option named WITH_FMT_VERSION

so we can optionally force cmake to use the fmt submodule even if
fmtlib is available in system.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #49452 from tchaikov/wip-crimson-test-fmt-v9
Kefu Chai [Fri, 16 Dec 2022 07:46:59 +0000 (15:46 +0800)]
Merge pull request #49452 from tchaikov/wip-crimson-test-fmt-v9

test/crimson: fixes related to fmtlib v9

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agomgr/dashboard: fix server side encryption config error 49173/head
Aashish Sharma [Thu, 1 Dec 2022 10:32:14 +0000 (16:02 +0530)]
mgr/dashboard: fix server side encryption config error

Fixes: https://tracker.ceph.com/issues/58296
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2 years agotest/crimson: include fmt/os.h 49452/head
Kefu Chai [Thu, 15 Dec 2022 10:23:43 +0000 (18:23 +0800)]
test/crimson: include fmt/os.h

so we can format std::error_code

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/crimson: print conn instead of its address
Kefu Chai [Thu, 15 Dec 2022 09:38:07 +0000 (17:38 +0800)]
test/crimson: print conn instead of its address

now that crimson::net::Connection can be formatted using fmt::format(),
instead of printing the pointer, let's print its value.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/crimson: trade operator<<() for fmt::format<>()
Kefu Chai [Thu, 15 Dec 2022 09:36:05 +0000 (17:36 +0800)]
test/crimson: trade operator<<() for fmt::format<>()

so we are able to format Payload using fmtlib v9. before fmtlib v9,
fmtlib is able to fall back to the operator<<() to format the the
types with the operator<<() defined. but after fmtlib v9, we need to
explicitly define them for accessing the specialized formatter.

now that we have specialized fmt::format<> for Payload, and the only
consumer of operator<<() is the fmtlib, we can safely drop the latter.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/crimson: specialize fmt::formater<> for types to be formatted
Kefu Chai [Thu, 15 Dec 2022 09:32:55 +0000 (17:32 +0800)]
test/crimson: specialize fmt::formater<> for types to be formatted

since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<> is specialized for some types to be
formatted with fmtlib, so the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agomsg/msg_types.h: include fmt/format.h
Kefu Chai [Thu, 15 Dec 2022 08:20:50 +0000 (16:20 +0800)]
msg/msg_types.h: include fmt/format.h

for accessing the macro definition of FMT_VERSION, so we can have
definition of fmt::format<> if required.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/crimson: print pointer using fmt::ptr()
Kefu Chai [Thu, 15 Dec 2022 07:26:46 +0000 (15:26 +0800)]
test/crimson: print pointer using fmt::ptr()

in fmtlib v9, it does not format typed pointers anymore, instead,
we need to print them using fmt::ptr().

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agotest/crimson: include system and 3rd-party headers first
Kefu Chai [Thu, 15 Dec 2022 05:34:16 +0000 (13:34 +0800)]
test/crimson: include system and 3rd-party headers first

see also https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #49467 from zdover23/wip-doc-2022-12-16-rados-ec-default-simplest
zdover23 [Fri, 16 Dec 2022 01:28:38 +0000 (11:28 +1000)]
Merge pull request #49467 from zdover23/wip-doc-2022-12-16-rados-ec-default-simplest

doc/rados: clarify default EC pool from simplest

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: clarify default EC pool from simplest 49467/head
Zac Dover [Fri, 16 Dec 2022 01:07:51 +0000 (11:07 +1000)]
doc/rados: clarify default EC pool from simplest

Distinguish between the default erasure-coded pool (k=2 m=2) and the
simplest erasure-coded pool (k=2 m=1).

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #48991 from kamoltat/wip-ksirivad-fix-bz-2121452
Kamoltat Sirivadhna [Thu, 15 Dec 2022 22:07:38 +0000 (17:07 -0500)]
Merge pull request #48991 from kamoltat/wip-ksirivad-fix-bz-2121452

mon/Elector: Change how we handle removed_ranks and notify_rank_removed()
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2 years agoMerge pull request #49436 from Thingee/add-alex-to-board
Mike Perez [Thu, 15 Dec 2022 21:30:07 +0000 (13:30 -0800)]
Merge pull request #49436 from Thingee/add-alex-to-board

doc/foundation: Adding Alex Marangone to the board

2 years agoMerge pull request #49416 from ljflores/wip-telemetry-label
Laura Flores [Thu, 15 Dec 2022 17:04:27 +0000 (11:04 -0600)]
Merge pull request #49416 from ljflores/wip-telemetry-label

.github: automatically add telemetry label to relevant files

2 years agoMerge pull request #48641 from NitzanMordhai/wip-nitzan-valgrind-supp-adding-rocksdb...
Yuri Weinstein [Thu, 15 Dec 2022 16:14:03 +0000 (08:14 -0800)]
Merge pull request #48641 from NitzanMordhai/wip-nitzan-valgrind-supp-adding-rocksdb-dl-init

src/valgrind.supp: Adding know leaks unrelated to ceph

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #48416 from Rethan/wip-bluestore-discard-thread
Yuri Weinstein [Thu, 15 Dec 2022 16:13:33 +0000 (08:13 -0800)]
Merge pull request #48416 from Rethan/wip-bluestore-discard-thread

blk/KernelDevice: don't start discard thread if device not support_di…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2 years agoMerge pull request #40066 from ideepika/wip-mempool-buffers
Yuri Weinstein [Thu, 15 Dec 2022 16:13:06 +0000 (08:13 -0800)]
Merge pull request #40066 from ideepika/wip-mempool-buffers

os/bluestore: improve cache_onode and cache_buffer mempool accounting

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/cephfs: simplify ceph fs volume create command 49203/head
Ken Dreyer [Fri, 2 Dec 2022 18:10:19 +0000 (13:10 -0500)]
doc/cephfs: simplify ceph fs volume create command

Make it easy for new users to choose a sensible default "cephfs"
filesystem name.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2 years agoMerge pull request #49435 from dang/wip-dang-motr-build
Daniel Gryniewicz [Thu, 15 Dec 2022 14:57:51 +0000 (09:57 -0500)]
Merge pull request #49435 from dang/wip-dang-motr-build

RGW - Fix DAOS and MOTR stores to not use rgw_obj_key::to_str()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge PR #49021 into main
Venky Shankar [Thu, 15 Dec 2022 11:21:23 +0000 (16:51 +0530)]
Merge PR #49021 into main

* refs/pull/49021/head:
qa: switch to https protocol for repos' server

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2 years agoMerge pull request #48707 from guits/guits-tracker-57918
Guillaume Abrioux [Thu, 15 Dec 2022 06:59:11 +0000 (07:59 +0100)]
Merge pull request #48707 from guits/guits-tracker-57918

ceph-volume: fix a bug in get_all_devices_vgs()

2 years agoMerge pull request #49445 from zdover23/wip-doc-2022-12-15-cephfs-posix-eg-fix
colemitchell [Thu, 15 Dec 2022 04:36:30 +0000 (23:36 -0500)]
Merge pull request #49445 from zdover23/wip-doc-2022-12-15-cephfs-posix-eg-fix

doc/cephfs: fix "e.g." in posix.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/cephfs: fix "e.g." in posix.rst 49445/head
Zac Dover [Thu, 15 Dec 2022 04:08:20 +0000 (14:08 +1000)]
doc/cephfs: fix "e.g." in posix.rst

Remove "e.g." and replace it with "for example".

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49441 from zdover23/wip-doc-2022-12-15-cephfs-homophone-error
colemitchell [Thu, 15 Dec 2022 04:06:11 +0000 (23:06 -0500)]
Merge pull request #49441 from zdover23/wip-doc-2022-12-15-cephfs-homophone-error

doc/cephfs - s/yet to here/yet to hear/ posix.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #49442 from zdover23/wip-doc-2022-12-15-cephfs-homophone-error-2
colemitchell [Thu, 15 Dec 2022 04:05:48 +0000 (23:05 -0500)]
Merge pull request #49442 from zdover23/wip-doc-2022-12-15-cephfs-homophone-error-2

doc/cephfs: s/all of there are/all of these are/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #48089 from aisakaki/wip-omap-rmkeyrange
Yingxin [Thu, 15 Dec 2022 02:37:50 +0000 (10:37 +0800)]
Merge pull request #48089 from aisakaki/wip-omap-rmkeyrange

crimson/os/seastore: introduce omap range remove

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #49427 from tchaikov/wip-crimson-fmt
Yingxin [Thu, 15 Dec 2022 02:24:17 +0000 (10:24 +0800)]
Merge pull request #49427 from tchaikov/wip-crimson-fmt

crimson: fmtlib v9 related changes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #49408 from tchaikov/wip-crimson-device_spec_t
Yingxin [Thu, 15 Dec 2022 02:21:22 +0000 (10:21 +0800)]
Merge pull request #49408 from tchaikov/wip-crimson-device_spec_t

crimson/os: define device_spec_t as packed

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agodoc/cephfs: s/all of there are/all of these are/ 49442/head
Zac Dover [Wed, 14 Dec 2022 21:07:44 +0000 (07:07 +1000)]
doc/cephfs: s/all of there are/all of these are/

s/all of there are/all of these are/ in posix.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agodoc/cephfs - s/yet to here/yet to hear/ posix.rst 49441/head
Zac Dover [Wed, 14 Dec 2022 20:51:29 +0000 (06:51 +1000)]
doc/cephfs - s/yet to here/yet to hear/ posix.rst

Correct "yet to here" to "yet to hear" in posix.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #48710 from cbodley/wip-rgw-process-env
Casey Bodley [Wed, 14 Dec 2022 20:15:46 +0000 (15:15 -0500)]
Merge pull request #48710 from cbodley/wip-rgw-process-env

rgw: pass RGWProcessEnv into process_request()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #49395 from adamemerson/wip-policy-useful-error-messages
Adam C. Emerson [Wed, 14 Dec 2022 20:13:51 +0000 (15:13 -0500)]
Merge pull request #49395 from adamemerson/wip-policy-useful-error-messages

rgw: Give useful errors when policies fail to parse

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agomon/Monitor.cc: notify_new_monmap() skips removal of non-exist rank 48991/head
Kamoltat [Wed, 14 Dec 2022 19:54:00 +0000 (19:54 +0000)]
mon/Monitor.cc: notify_new_monmap() skips removal of non-exist rank

Problem:
In RHCS the user can choose to manually remove a monitor rank
before shutting the monitor down. Causing inconsistency in monmap.
for example we remove mon.a from the monmap, there is a short period
where mon.a is still operational and will try to remove itself from
monmap but we will run into an assertion in
ConnectionTracker::notify_ranks_removed().

Solution:
In Monitor::notify_new_monmap() we prevent the func
from going into removing our own rank, or
ranks that doesn't exists in monmap.

FYI: this is an RHCS problem only, in ODF,
we never remove a monitor from monmap
before shutting it down.

Fixes: https://tracker.ceph.com/issues/58049
Signed-off-by: Kamoltat <ksirivad@redhat.com>
2 years agoMerge pull request #49141 from cbodley/wip-58111
Casey Bodley [Wed, 14 Dec 2022 19:59:58 +0000 (14:59 -0500)]
Merge pull request #49141 from cbodley/wip-58111

rgw: rgw_parse_url_bucket() rejects empty bucket names after 'tenant:'

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@cern.ch>
2 years agoMerge pull request #48226 from sseshasa/wip-fix-recovery-backfill-limits-for-mclock
Neha Ojha [Wed, 14 Dec 2022 18:59:15 +0000 (10:59 -0800)]
Merge pull request #48226 from sseshasa/wip-fix-recovery-backfill-limits-for-mclock

osd: Reduce backfill/recovery default limits for mClock and other optimizations

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agodoc/foundation: Adding Alex Marangone to the board 49436/head
Mike Perez [Wed, 14 Dec 2022 18:57:50 +0000 (10:57 -0800)]
doc/foundation: Adding Alex Marangone to the board

Signed-off-by: Mike Perez <thingee@gmail.com>
2 years agoRGW - Fix DAOS and MOTR stores to not use rgw_obj_key::to_str() 49435/head
Daniel Gryniewicz [Wed, 14 Dec 2022 18:45:31 +0000 (13:45 -0500)]
RGW - Fix DAOS and MOTR stores to not use rgw_obj_key::to_str()

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoceph-volume: fix a bug in get_all_devices_vgs() 48707/head
Guillaume Abrioux [Wed, 2 Nov 2022 15:03:17 +0000 (16:03 +0100)]
ceph-volume: fix a bug in get_all_devices_vgs()

When a physical volume isn't a member of any volume group,
the function `get_all_devices_vgs()` in `ceph_volume.api.lvm`
shouldn't try to return an `VolumeGroup()` object against this
device.

Fixes: https://tracker.ceph.com/issues/57918
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years ago.github: automatically add telemetry label to relevant files 49416/head
Laura Flores [Tue, 13 Dec 2022 20:01:11 +0000 (20:01 +0000)]
.github: automatically add telemetry label to relevant files

Signed-off-by: Laura Flores <ljflores@redhat.com>
2 years agoMerge pull request #49178 from JoshSalomon/chg-primary
Laura Flores [Wed, 14 Dec 2022 15:52:55 +0000 (09:52 -0600)]
Merge pull request #49178 from JoshSalomon/chg-primary

osd: Add cli upmap command to change the primary OSD of a pg

2 years agoMerge pull request #48742 from rhcs-dashboard/add-tooltip
Pere Diaz Bou [Wed, 14 Dec 2022 14:06:09 +0000 (15:06 +0100)]
Merge pull request #48742 from rhcs-dashboard/add-tooltip

mgr/dashboard: add tooltip mirroring pools table

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #49418 from tchaikov/wip-cls-fifo-fmt
Kefu Chai [Wed, 14 Dec 2022 13:51:56 +0000 (21:51 +0800)]
Merge pull request #49418 from tchaikov/wip-cls-fifo-fmt

cls/fifo: specialize fmt::formater<> for rados::cls::fifo::info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agomgr/dashboard: display real health in rbd mirroring pools 47276/head
Pere Diaz Bou [Tue, 26 Jul 2022 09:56:57 +0000 (11:56 +0200)]
mgr/dashboard: display real health in rbd mirroring pools

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoosd, mon: Add 'osd pg-upmap-primary' and 'osd rm-pg-upmap-primary' commands. 49178/head
Josh Salomon [Wed, 2 Nov 2022 14:23:28 +0000 (16:23 +0200)]
osd, mon: Add 'osd pg-upmap-primary' and 'osd rm-pg-upmap-primary' commands.

Changed OSDMap to keep upmap-primary records
The upmap-primary records are visible in json/xml and in dump commands.

Signed-off-by: Josh Salomon <jsalomon@redhat.com>