]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocrimson: keep do_osd_ops_params_t alive when call do_osd_ops 45820/head
chunmei-liu [Fri, 8 Apr 2022 00:35:37 +0000 (17:35 -0700)]
crimson: keep do_osd_ops_params_t alive when call do_osd_ops

otherwise stack-under-overflow

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #45774 from liu-chunmei/crimson-cmp-xattr
Samuel Just [Tue, 5 Apr 2022 19:33:01 +0000 (12:33 -0700)]
Merge pull request #45774 from liu-chunmei/crimson-cmp-xattr

crimson: cmp xattr canceled operation

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #45740 from cbodley/wip-doc-cmake-bundled-deps
Casey Bodley [Tue, 5 Apr 2022 14:19:57 +0000 (10:19 -0400)]
Merge pull request #45740 from cbodley/wip-doc-cmake-bundled-deps

README: document cmake options for system packages

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #45743 from m-ildefons/docs-python3-librados 45778/head
Kefu Chai [Tue, 5 Apr 2022 05:36:38 +0000 (13:36 +0800)]
Merge pull request #45743 from m-ildefons/docs-python3-librados

docs: update install steps for python librados

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agocrimson: store->get_attr return nodata, cancel following operations 45774/head
chunmei-liu [Tue, 5 Apr 2022 04:14:14 +0000 (21:14 -0700)]
crimson: store->get_attr return nodata, cancel following operations

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agocrimson: if do_cmp_xattr failed, cancel the following operations
chunmei-liu [Tue, 5 Apr 2022 04:10:33 +0000 (21:10 -0700)]
crimson: if do_cmp_xattr failed, cancel the following operations

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #45764 from zdover23/wip-doc-2022-04-04-os-recommendations
zdover23 [Mon, 4 Apr 2022 20:54:41 +0000 (06:54 +1000)]
Merge pull request #45764 from zdover23/wip-doc-2022-04-04-os-recommendations

doc/start: remove old OS tables

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #45720 from xxhdx1985126/wip-55143
Samuel Just [Mon, 4 Apr 2022 20:02:10 +0000 (13:02 -0700)]
Merge pull request #45720 from xxhdx1985126/wip-55143

crimson/os/seastore: use different segment seq allocator from journal and ool segments

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #45735 from chrigl/fix-54620
Adam King [Mon, 4 Apr 2022 19:17:26 +0000 (15:17 -0400)]
Merge pull request #45735 from chrigl/fix-54620

mgr/cephadm: support non-root ssh-user w permissions

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45710 from rkachach/fix_issue_55132
Adam King [Mon, 4 Apr 2022 19:15:33 +0000 (15:15 -0400)]
Merge pull request #45710 from rkachach/fix_issue_55132

mgr/cephadm: fixing public network conf parsing

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45733 from benhanokh/simple_bitmap_fix
Neha Ojha [Mon, 4 Apr 2022 18:33:16 +0000 (11:33 -0700)]
Merge pull request #45733 from benhanokh/simple_bitmap_fix

os/BlueStore: NCB fix for SimpleBitmap boundary check

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
3 years agoMerge pull request #45349 from guits/check-check-device-is-available
Adam King [Mon, 4 Apr 2022 16:03:29 +0000 (12:03 -0400)]
Merge pull request #45349 from guits/check-check-device-is-available

ceph-volume: various fixes in arg_validators

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45687 from yuvalif/wip-yuval-multisite-test-rgws
Yuval Lifshitz [Mon, 4 Apr 2022 15:23:22 +0000 (18:23 +0300)]
Merge pull request #45687 from yuvalif/wip-yuval-multisite-test-rgws

multisite/test: running the multisite test script with multiple RGWs per zone

3 years agoMerge pull request #45696 from cbodley/wip-cmake-system-utf8proc
Casey Bodley [Mon, 4 Apr 2022 14:16:13 +0000 (10:16 -0400)]
Merge pull request #45696 from cbodley/wip-cmake-system-utf8proc

cmake: WITH_SYSTEM_UTF8PROC defaults to OFF

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
3 years agoMerge pull request #45763 from rhcs-dashboard/fix-55172-master
Ernesto Puerta [Mon, 4 Apr 2022 13:21:39 +0000 (15:21 +0200)]
Merge pull request #45763 from rhcs-dashboard/fix-55172-master

monitoring/grafana: fix version

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #45514 from tchaikov/wip-cxx20-fixes-cephfs
Venky Shankar [Mon, 4 Apr 2022 13:18:52 +0000 (18:48 +0530)]
Merge pull request #45514 from tchaikov/wip-cxx20-fixes-cephfs

cephfs: C++20 related fixes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agodoc/start: remove old OS tables 45764/head
Zac Dover [Mon, 4 Apr 2022 13:18:27 +0000 (23:18 +1000)]
doc/start: remove old OS tables

This PR removes tables that contained outdated
information about Ceph's supported operating
systems. Ernesto Puerta had the idea to consolidate
the information that was in those tables into a single
table, and we're doing that.

Part of that process involves removing the old tables.
That's what this PR does.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agomgr/cephadm: fixing public network conf parsing 45710/head
Redouane Kachach [Wed, 30 Mar 2022 13:48:40 +0000 (15:48 +0200)]
mgr/cephadm: fixing public network conf parsing
Fixes: https://tracker.ceph.com/issues/55132
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
3 years agomonitoring/grafana: fix version 45763/head
Ernesto Puerta [Mon, 4 Apr 2022 11:52:43 +0000 (13:52 +0200)]
monitoring/grafana: fix version

Fixes: https://tracker.ceph.com/issues/55172
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
3 years agoMerge pull request #45254 from travisn/prometheus-rules-typos
Ernesto Puerta [Mon, 4 Apr 2022 11:46:00 +0000 (13:46 +0200)]
Merge pull request #45254 from travisn/prometheus-rules-typos

prometheus: Spell check the alert descriptions

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
3 years agoMerge pull request #45746 from idryomov/wip-crush-rule-sizes-cleanup
Ilya Dryomov [Mon, 4 Apr 2022 11:31:05 +0000 (13:31 +0200)]
Merge pull request #45746 from idryomov/wip-crush-rule-sizes-cleanup

doc,test: clean up crush rule min/max_size leftovers

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #45385 from zdover23/wip-doc-2022-03-14-os-recommendations
zdover23 [Mon, 4 Apr 2022 04:20:12 +0000 (14:20 +1000)]
Merge pull request #45385 from zdover23/wip-doc-2022-03-14-os-recommendations

doc/start: update os-recommendations.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
3 years agoMerge pull request #45705 from SUSE/wip-really-fix-55079
Tim Serong [Mon, 4 Apr 2022 03:49:32 +0000 (13:49 +1000)]
Merge pull request #45705 from SUSE/wip-really-fix-55079

ceph.spec.in: remove build directory in %clean, not %install

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44612 from rzarzynski/wip-bs-lazy4freebsd
Radoslaw Zarzynski [Sat, 2 Apr 2022 09:44:37 +0000 (11:44 +0200)]
Merge pull request #44612 from rzarzynski/wip-bs-lazy4freebsd

bdev: fix FTBFS on FreeBSD, keep the huge paged read buffers.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #45719 from liu-chunmei/crimson-setopflag
Liu-Chunmei [Sat, 2 Apr 2022 04:01:29 +0000 (21:01 -0700)]
Merge pull request #45719 from liu-chunmei/crimson-setopflag

Crimson handle CEPH_OSD_OP_FLAG_FAILOK

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocrimson/os/seastore: use different segment seq allocator from journal and ool segments 45720/head
Xuehan Xu [Thu, 31 Mar 2022 05:48:42 +0000 (13:48 +0800)]
crimson/os/seastore: use different segment seq allocator from journal and ool segments

Fixes: https://tracker.ceph.com/issues/55143
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore: rename segment_header_t::journal_segment_seq to segment_seq
Xuehan Xu [Thu, 31 Mar 2022 02:50:42 +0000 (10:50 +0800)]
crimson/os/seastore: rename segment_header_t::journal_segment_seq to segment_seq

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agocrimson/os/seastore/cache: clean up in Cache::get_next_dirty_extents()
Xuehan Xu [Thu, 31 Mar 2022 02:22:05 +0000 (10:22 +0800)]
crimson/os/seastore/cache: clean up in Cache::get_next_dirty_extents()

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
3 years agoMerge pull request #45658 from ceph/wip-debug-install
David Galloway [Fri, 1 Apr 2022 19:25:41 +0000 (15:25 -0400)]
Merge pull request #45658 from ceph/wip-debug-install

build: Add some debugging messages

3 years agobuild: Add some debugging messages 45658/head
David Galloway [Fri, 25 Mar 2022 21:29:44 +0000 (17:29 -0400)]
build: Add some debugging messages

Having a unique string like "CI_DEBUG" will help me know where we are in the build process in Jenkins logs.

Signed-off-by: David Galloway <dgallowa@redhat.com>
3 years agoMerge pull request #45739 from rhcs-dashboard/fix-55155-master
David Galloway [Fri, 1 Apr 2022 17:30:05 +0000 (13:30 -0400)]
Merge pull request #45739 from rhcs-dashboard/fix-55155-master

grafana/Makefile: don't push to docker

3 years agodoc/start: update os-recommendations.rst 45385/head
Zac Dover [Mon, 14 Mar 2022 21:20:40 +0000 (07:20 +1000)]
doc/start: update os-recommendations.rst

This PR updates the extremely out-of-date
os-recommendations.rst file.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agosrc/crush: remove sample.txt 45746/head
Ilya Dryomov [Mon, 28 Mar 2022 18:51:44 +0000 (20:51 +0200)]
src/crush: remove sample.txt

It is hopelessly outdated -- the weights were moved from crushmap to
osdmap in 2008, etc.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agodoc,test: clean up crush rule min/max_size leftovers
Ilya Dryomov [Mon, 28 Mar 2022 18:35:35 +0000 (20:35 +0200)]
doc,test: clean up crush rule min/max_size leftovers

These fields were removed in quincy in commit f95eb04411cd ("crush:
eliminate min_size and max_size").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agografana/Makefile: don't push to docker 45739/head
Ernesto Puerta [Thu, 31 Mar 2022 18:29:17 +0000 (20:29 +0200)]
grafana/Makefile: don't push to docker

Fixes: https://tracker.ceph.com/issues/55155
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
3 years agoMerge pull request #45583 from p-se/monitoring-alert-mtu-group-by-devices
Ernesto Puerta [Fri, 1 Apr 2022 09:11:30 +0000 (11:11 +0200)]
Merge pull request #45583 from p-se/monitoring-alert-mtu-group-by-devices

mgr/dashboard: Compare values of MTU alert by device

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
3 years agodocs: update install steps for python librados 45743/head
Moritz Röhrich [Fri, 1 Apr 2022 08:36:53 +0000 (10:36 +0200)]
docs: update install steps for python librados

- Put the correct package names for python librados in the install
  instructions.
- librados-devel does not actually pull in python3-librados as
  dependency anymore, remove mention of this.
- Add instructions for openSUSE/SLE

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
3 years agoMerge pull request #45742 from tchaikov/wip-python-debian
Kefu Chai [Fri, 1 Apr 2022 06:57:41 +0000 (14:57 +0800)]
Merge pull request #45742 from tchaikov/wip-python-debian

debian: install from site-packages to dist-packages

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agodebian: install from site-packages to dist-packages 45742/head
Kefu Chai [Fri, 1 Apr 2022 00:22:11 +0000 (08:22 +0800)]
debian: install from site-packages to dist-packages

this change addresses the regression introduced by
637dd7b40404e644519b1fc3b5ef03f2d18def00

dist-packages is a debian specific directory for holding 3rd party
python modules. so install the pure python packages into it.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agocrimson: set result = 0 when op.flags include CEPH_OSD_OP_FLAG_FAILOK 45719/head
Chunmei Liu [Wed, 30 Mar 2022 00:26:15 +0000 (17:26 -0700)]
crimson: set result = 0 when op.flags include CEPH_OSD_OP_FLAG_FAILOK

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocrimson: handle osd_op.op.flags include CEPH_OSD_OP_FLAG_FAILOK
chunmei-liu [Thu, 31 Mar 2022 22:14:14 +0000 (15:14 -0700)]
crimson: handle osd_op.op.flags include CEPH_OSD_OP_FLAG_FAILOK

If the osd_op failed, still continue to next op when it include
CEPH_OSD_OP_FLAG_FAILOK.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
3 years agoMerge pull request #45707 from rzarzynski/wip-crimson-errorator-2nd-arg-handle
Liu-Chunmei [Thu, 31 Mar 2022 22:15:12 +0000 (15:15 -0700)]
Merge pull request #45707 from rzarzynski/wip-crimson-errorator-2nd-arg-handle

crimson/common: let errorator's handlers to conditionally forward errors

Reviewed-by: Sam Just <sjust@redhat.com>
                       liu-chunmei <chunmei.liu@intel.com>

3 years agoREADME: document cmake options for system packages 45740/head
Casey Bodley [Thu, 31 Mar 2022 18:35:45 +0000 (14:35 -0400)]
README: document cmake options for system packages

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45578 from rhcs-dashboard/fix-grafana-build
Ernesto Puerta [Thu, 31 Mar 2022 17:58:29 +0000 (19:58 +0200)]
Merge pull request #45578 from rhcs-dashboard/fix-grafana-build

mgr/dashboard: remove transition-through-oci image workaround in grafana  build

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoos/BlueStore: NCB fix for SimpleBitmap boundary check 45733/head
Gabriel BenHanokh [Thu, 31 Mar 2022 11:47:59 +0000 (14:47 +0300)]
os/BlueStore: NCB fix for SimpleBitmap boundary check
The boundary check in SimpleBitmap is off by one causing an assert to trigger
Also fixed a bug when asking for the next clear_extent on a unaligned map when the last bits in the map were set.
Adding unit-tests
Fixes: https://tracker.ceph.com/issues/55145
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
3 years agomgr/cephadm: support non-root ssh-user w permissions 45735/head
Christoph Glaubitz [Thu, 31 Mar 2022 12:57:53 +0000 (12:57 +0000)]
mgr/cephadm: support non-root ssh-user w permissions

Restructured code, so that in case of non-root, the resulting file will
be created with permissions set to the ssh-user. This allows the
subsequent scp to be able to write the file. The remaining code kept the
same, so that file permissions are restored to the expected ones, but
just runs after the scp.

Fixes: https://tracker.ceph.com/issues/54620
Signed-off-by: Christoph Glaubitz <c.glaubitz@syseleven.de>
3 years agobdev: fix FTBFS on FreeBSD, keep the huge paged read buffers. 44612/head
Radoslaw Zarzynski [Mon, 17 Jan 2022 14:55:05 +0000 (14:55 +0000)]
bdev: fix FTBFS on FreeBSD, keep the huge paged read buffers.

Special thanks to Willem Jan Withagen!

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #45660 from CongMinYin/fix-pwl-status
Ilya Dryomov [Thu, 31 Mar 2022 10:30:03 +0000 (12:30 +0200)]
Merge pull request #45660 from CongMinYin/fix-pwl-status

librbd/cache/pwl: correct cache state

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoprometheus: spell check the alert descriptions 45254/head
Travis Nielsen [Fri, 4 Mar 2022 17:42:02 +0000 (10:42 -0700)]
prometheus: spell check the alert descriptions

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
3 years agocrimson/common: let errorator's handlers to conditionally forward errors 45707/head
Radosław Zarzyński [Wed, 30 Mar 2022 13:02:20 +0000 (15:02 +0200)]
crimson/common: let errorator's handlers to conditionally forward errors

The use case is like the following one:

```diff
diff --git a/src/crimson/osd/replicated_recovery_backend.cc b/src/crimson/osd/replicated_recovery_backend.cc
index a96f1b8dd2b..055add2e407 100644
--- a/src/crimson/osd/replicated_recovery_backend.cc
+++ b/src/crimson/osd/replicated_recovery_backend.cc
@@ -516,10 +516,14 @@ ReplicatedRecoveryBackend::read_object_for_push_op(
       recovered_to = push_op->data_included.range_end();
     }
     return seastar::make_ready_future<uint64_t>(recovered_to);
-  }, PGBackend::read_errorator::all_same_way([](auto e) {
+  }, PGBackend::read_errorator::all_same_way([](auto e, auto&& e_raw) -> PGBackend::read_errorator::future<uint64_t> {
     logger().debug("build_push_op: read exception");
-    return seastar::make_exception_future<uint64_t>(e);
-  }));
+    if (42) {
+      return std::move(e_raw);
+    } else {
+      return seastar::make_exception_future<uint64_t>(e);
+    }
+  })).handle_error_interruptible(PGBackend::read_errorator::discard_all{});
 }

 RecoveryBackend::interruptible_future<>
```

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agoMerge pull request #45361 from adk3798/mds-upgrade-test
Adam King [Wed, 30 Mar 2022 22:56:45 +0000 (18:56 -0400)]
Merge pull request #45361 from adk3798/mds-upgrade-test

qa/suites/fs: stop looping in mds upgrade test if upgrade failed

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoceph-volume/tests: reject loop devices in lvm.conf 45349/head
Guillaume Abrioux [Wed, 30 Mar 2022 14:18:26 +0000 (16:18 +0200)]
ceph-volume/tests: reject loop devices in lvm.conf

The current task doesn't works (typo?).
Otherwise api/lvm.py can't work properly, functions such as
`get_single_lv()` and many other don't return the expected results.
Indeed, lvm is confused because of the nvme_loop setup.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume: do not leave pv when zapping osds
Guillaume Abrioux [Mon, 28 Mar 2022 22:01:39 +0000 (00:01 +0200)]
ceph-volume: do not leave pv when zapping osds

when zapping a device and no vg/lv are left, the pv should be
removed too.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoorchestrator: support complex osd creation
Guillaume Abrioux [Wed, 23 Mar 2022 09:04:45 +0000 (10:04 +0100)]
orchestrator: support complex osd creation

This adds the support of complex OSD creation with command
`orch daemon add osd`.
Any argument supported by `DriveGroupSpec()` can be passed on the command line.

Usage:
```
ceph orch daemon add osd host:data_devices=device1,device2,db_devices=device3,osds_per_device=2,...
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoDriveSelection: skip unavailable devices
Guillaume Abrioux [Tue, 22 Mar 2022 15:35:58 +0000 (16:35 +0100)]
DriveSelection: skip unavailable devices

Cephadm shouldn't try to deploy a disk reported as unavailable by ceph-volume.
The idea here is to check the rejection reason so we can still use DB devices
in case of OSD replacement.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume: various fixes in arg_validators
Guillaume Abrioux [Fri, 11 Mar 2022 09:29:35 +0000 (10:29 +0100)]
ceph-volume: various fixes in arg_validators

if a device with an FS is passed, ceph-volume should abort
the OSD creation.

Fixes: https://tracker.ceph.com/issues/54535
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agodoc/cephadm: fix a typo
Guillaume Abrioux [Wed, 23 Mar 2022 09:07:05 +0000 (10:07 +0100)]
doc/cephadm: fix a typo

s/osd_crush_choose_leaf_type/osd_crush_chooseleaf_type

```
[ceph: root@adm-1 /]# ceph config set global osd_crush_choose_leaf_type 0
Error EINVAL: unrecognized config option 'osd_crush_choose_leaf_type'
[ceph: root@adm-1 /]# ceph config set global osd_crush_chooseleaf_type 0
[ceph: root@adm-1 /]#
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume/tests: speed up tox tests
Guillaume Abrioux [Mon, 14 Mar 2022 14:40:47 +0000 (14:40 +0000)]
ceph-volume/tests: speed up tox tests

Let's use `--numprocesses=auto` in order to speed up the unit tests execution.

See the difference, without `--numprocesses=auto`:
```

... omitted output ...

real    1m22.884s
user    0m23.003s
sys     0m20.504s
```

with `--numprocesses=auto`:

```

... omitted output ...

real    0m18.767s
user    0m33.056s
sys     0m23.244s
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoMerge pull request #45248 from adk3798/offline-thread
Adam King [Wed, 30 Mar 2022 17:50:07 +0000 (13:50 -0400)]
Merge pull request #45248 from adk3798/offline-thread

mgr/cephadm: offline host watcher

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #45344 from adk3798/raw-prepare-cmd
Adam King [Wed, 30 Mar 2022 17:48:43 +0000 (13:48 -0400)]
Merge pull request #45344 from adk3798/raw-prepare-cmd

mgr/cephadm: generate one c-v raw prepare cmd per data device in raw mode

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #45567 from cbodley/wip-55000
Casey Bodley [Wed, 30 Mar 2022 16:24:56 +0000 (12:24 -0400)]
Merge pull request #45567 from cbodley/wip-55000

rgw multisite: data sync uses yield_spawn_window()

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #45515 from tchaikov/wip-cxx20-fixes-rgw
Casey Bodley [Wed, 30 Mar 2022 16:03:15 +0000 (12:03 -0400)]
Merge pull request #45515 from tchaikov/wip-cxx20-fixes-rgw

rgw: C++20 related fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45536 from tchaikov/wip-std-optional
Kefu Chai [Wed, 30 Mar 2022 15:16:15 +0000 (23:16 +0800)]
Merge pull request #45536 from tchaikov/wip-std-optional

compressor/QatAccel: s/boost::optional/std::optional/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45648 from tchaikov/wip-kill-distutils
Kefu Chai [Wed, 30 Mar 2022 14:56:32 +0000 (22:56 +0800)]
Merge pull request #45648 from tchaikov/wip-kill-distutils

cmake: avoid using distutils

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #45694 from amathuria/amathuri-53923-fix
Yuri Weinstein [Wed, 30 Mar 2022 14:16:44 +0000 (07:16 -0700)]
Merge pull request #45694 from amathuria/amathuri-53923-fix

osd/osd_types: Increasing decode version of scrub_duration in pg stats

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45142 from SoftIron/ceph-mgr-cephadm-python3-cherrypyTHREE
Kefu Chai [Wed, 30 Mar 2022 13:57:19 +0000 (21:57 +0800)]
Merge pull request #45142 from SoftIron/ceph-mgr-cephadm-python3-cherrypyTHREE

debian/control: fix python3-cherrypy*3* dependency

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agolibrbd/cache/pwl: correct cache state 45660/head
Yin Congmin [Tue, 28 Dec 2021 06:10:35 +0000 (14:10 +0800)]
librbd/cache/pwl: correct cache state

update cache state after dirty_entries or log_enties list updated.

Fixes: https://tracker.ceph.com/issues/50614
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years agoMerge pull request #45217 from rkachach/fix_issue_47872
Adam King [Wed, 30 Mar 2022 13:01:40 +0000 (09:01 -0400)]
Merge pull request #45217 from rkachach/fix_issue_47872

mgr/cephadm: check spec host when adding osd

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45275 from rkachach/fix_issue_54342
Adam King [Wed, 30 Mar 2022 12:59:15 +0000 (08:59 -0400)]
Merge pull request #45275 from rkachach/fix_issue_54342

mgr/cephadm: adding HostSpec validation

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45534 from rkachach/fix_issue_50804
Adam King [Wed, 30 Mar 2022 12:58:04 +0000 (08:58 -0400)]
Merge pull request #45534 from rkachach/fix_issue_50804

mgr/cephadm: Show warning when user provides --fsid option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
3 years agoMerge pull request #45692 from rkachach/fix_issue_55113
Adam King [Wed, 30 Mar 2022 12:48:33 +0000 (08:48 -0400)]
Merge pull request #45692 from rkachach/fix_issue_55113

mgr/cephadm: fallback to normal sorted if cannot import natsorted

Reviewed-by: Adam King <adking@redhat,com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agocrimson/common: avoid unnecessary throwing in stateful_error_t
Radosław Zarzyński [Wed, 30 Mar 2022 12:21:11 +0000 (14:21 +0200)]
crimson/common: avoid unnecessary throwing in stateful_error_t

This is an optimization for the flexible-but-slow `stateful_error_t`
error family of errorator. It's used infrequently.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agodebian/control: fix python3-cherrypy*3* dependency 45142/head
Koen Kooi [Wed, 23 Feb 2022 16:40:48 +0000 (08:40 -0800)]
debian/control: fix python3-cherrypy*3* dependency

The trailing '3' was missed in one instance, ceph-mgr-cephadm, leading to:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 ceph-mgr-cephadm : Depends: python3-cherrypy but it is not installable

Which makes the installation fail.

Fixes: 78983ad0d0cce422da32dc4876ac186f6d32c3f5
Signed-off-by: Koen Kooi <koen@softiron.com>
3 years agoMerge pull request #45335 from rhcs-dashboard/fix-54513-master
Ernesto Puerta [Wed, 30 Mar 2022 12:05:38 +0000 (14:05 +0200)]
Merge pull request #45335 from rhcs-dashboard/fix-54513-master

mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agoMerge pull request #45559 from rhcs-dashboard/existing-host-test-fix
Ernesto Puerta [Wed, 30 Mar 2022 11:00:36 +0000 (13:00 +0200)]
Merge pull request #45559 from rhcs-dashboard/existing-host-test-fix

mgr/dashboard: fix 01-hosts.e2e test error

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #45701 from inspur-wyq/wip-doc-typos
Anthony D'Atri [Wed, 30 Mar 2022 06:54:42 +0000 (23:54 -0700)]
Merge pull request #45701 from inspur-wyq/wip-doc-typos

doc/radosgw/sync-modules.rst: fix typos

3 years agoMerge pull request #45702 from inspur-wyq/wip-doc-my-typos
Anthony D'Atri [Wed, 30 Mar 2022 06:43:46 +0000 (23:43 -0700)]
Merge pull request #45702 from inspur-wyq/wip-doc-my-typos

doc/cephadm/operations.rst: fix typos

3 years agoceph.spec.in: remove build directory in %clean, not %install 45705/head
Tim Serong [Wed, 30 Mar 2022 05:25:30 +0000 (16:25 +1100)]
ceph.spec.in: remove build directory in %clean, not %install

Removing the build directory at the end of %install is too soon,
and means we get rid of a bunch of stuff needed to correctly
create debuginfo/debugsource packages, which happens automatically
right after %install.  So, let's put it where it really belongs, in
the %clean section.

Fixes: aa18cb12003e3526c8e8f23dc2335a483fbfa68e
Fixes: https://tracker.ceph.com/issues/55079
Signed-off-by: Tim Serong <tserong@suse.com>
3 years agoMerge pull request #45630 from liu-chunmei/crimson-omap-cmp
Samuel Just [Wed, 30 Mar 2022 04:40:49 +0000 (21:40 -0700)]
Merge pull request #45630 from liu-chunmei/crimson-omap-cmp

crimson: add omap cmp

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agodoc/cephadm/operations.rst: fix typos 45702/head
wangyunqing [Wed, 30 Mar 2022 03:53:57 +0000 (11:53 +0800)]
doc/cephadm/operations.rst: fix typos

Signed-off-by: wangyunqing <wangyunqing@inspur.com>
3 years agodoc/radosgw/sync-modules.rst: fix typos 45701/head
wangyunqing [Wed, 30 Mar 2022 03:53:57 +0000 (11:53 +0800)]
doc/radosgw/sync-modules.rst: fix typos

Signed-off-by: wangyunqing <wangyunqing@inspur.com>
3 years agoMerge pull request #45526 from xxhdx1985126/wip-coordinate-segment-seq
Samuel Just [Wed, 30 Mar 2022 03:53:30 +0000 (20:53 -0700)]
Merge pull request #45526 from xxhdx1985126/wip-coordinate-segment-seq

crimson/os/seastore: coordinate segment seq of journal and ool segments

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: return 0 in osd_op.rval when enodata 45630/head
Chunmei Liu [Thu, 24 Mar 2022 06:37:59 +0000 (23:37 -0700)]
crimson: return 0 in osd_op.rval when enodata

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocrimson: add OP_OMAP_CLEAR support
Chunmei Liu [Thu, 24 Mar 2022 06:59:39 +0000 (23:59 -0700)]
crimson: add OP_OMAP_CLEAR support

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocrimson:add OMAP_CMP support
Chunmei Liu [Sat, 19 Mar 2022 09:27:50 +0000 (02:27 -0700)]
crimson:add OMAP_CMP support

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agocmake: WITH_SYSTEM_UTF8PROC defaults to OFF 45696/head
Casey Bodley [Tue, 29 Mar 2022 21:46:04 +0000 (17:46 -0400)]
cmake: WITH_SYSTEM_UTF8PROC defaults to OFF

change the default value of WITH_SYSTEM_UTF8PROC from ON to OFF, so that
centos/rhel users can build with the default cmake configuration. no other
WITH_SYSTEM_* variable in ceph defaults to ON, so this is consistent
with other bundled libraries like boost and rocksdb

unfortunately, this also means that users that do have system packages
must opt-in to using them with -DWITH_SYSTEM_UTF8PROC=ON

both deb and rpm builds dependended on the previous default value, so
their logic was negated to match the new default

Fixes: https://tracker.ceph.com/issues/55114
Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agoosd/osd_types: Increasing decode version of scrub_duration in pg stats 45694/head
Aishwarya Mathuria [Tue, 29 Mar 2022 18:05:45 +0000 (23:35 +0530)]
osd/osd_types: Increasing decode version of scrub_duration in pg stats

All new fields added to pg stats after quincy RC need to have the decode field bumped up to avoid decoding errors during an upgrade from quincy RC to the quincy stable version

Fixes: https://tracker.ceph.com/issues/53923
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
3 years agomgr/cephadm: fallback to normal sorted if cannot import natsorted 45692/head
Redouane Kachach [Tue, 29 Mar 2022 16:37:10 +0000 (18:37 +0200)]
mgr/cephadm: fallback to normal sorted if cannot import natsorted
Fixes: https://tracker.ceph.com/issues/55113
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
3 years agoMerge pull request #45563 from rhcs-dashboard/fix-54991-master
Ernesto Puerta [Tue, 29 Mar 2022 17:03:49 +0000 (19:03 +0200)]
Merge pull request #45563 from rhcs-dashboard/fix-54991-master

mgr/dashboard:  don't log 3xx as errors

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agomultisite/test: running the multisite test script with multiple RGWs per zone 45687/head
Yuval Lifshitz [Tue, 29 Mar 2022 13:21:46 +0000 (16:21 +0300)]
multisite/test: running the multisite test script with multiple RGWs per zone

the RGW_PER_ZONE environment variables should be set to allow that
if not set, we assume 1 RGW per zone

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #45295 from lbausch/tweak_mgr_alerts_mail
Boris Ranto [Tue, 29 Mar 2022 11:55:14 +0000 (13:55 +0200)]
Merge pull request #45295 from lbausch/tweak_mgr_alerts_mail

mgr/alerts: Add Message-Id and Date header to sent emails

Reviewed-by: Boris Ranto <branto@redhat.com>
3 years agomgr/dashboard: don't log 3xx as errors 45563/head
Ernesto Puerta [Tue, 22 Mar 2022 13:40:30 +0000 (14:40 +0100)]
mgr/dashboard:  don't log 3xx as errors

Let's avoid printing these ugly/misleading/redundant messages:

```
0 [dashboard DEBUG controllers.home] frontend language from headers: ['en-us']
0 [dashboard DEBUG controllers.home] found directory for language 'en-us'
0 [dashboard DEBUG controllers.home] serving static content: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/src/pybind/mgr/dashboard/frontend/dist/en-US/styles.css
0 [dashboard ERROR exception] Internal Server Error
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/cherrypy/_cpdispatch.py", line 60, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/controllers/home.py", line 134, in __call__
    return serve_file(full_path)
  File "/usr/lib/python3/dist-packages/cherrypy/lib/static.py", line 70, in serve_file
    cptools.validate_since()
  File "/usr/lib/python3/dist-packages/cherrypy/lib/cptools.py", line 117, in validate_since
    raise cherrypy.HTTPRedirect([], 304)
cherrypy._cperror.HTTPRedirect: ([], 304)
```

Fixes: https://tracker.ceph.com/issues/54991
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
3 years agoMerge pull request #45676 from rzarzynski/wip-crimson-toolset-10
Kefu Chai [Tue, 29 Mar 2022 09:03:14 +0000 (17:03 +0800)]
Merge pull request #45676 from rzarzynski/wip-crimson-toolset-10

ceph.spec.in: use gcc-toolset-10 for building crimson

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44014 from rhcs-dashboard/host-detail-service-column
Ernesto Puerta [Tue, 29 Mar 2022 08:49:31 +0000 (10:49 +0200)]
Merge pull request #44014 from rhcs-dashboard/host-detail-service-column

mgr/dashboard: extend daemon actions to host details

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agoMerge pull request #45537 from yuvalif/wip-yuval-multisite-test-devices
Yuval Lifshitz [Tue, 29 Mar 2022 07:35:57 +0000 (10:35 +0300)]
Merge pull request #45537 from yuvalif/wip-yuval-multisite-test-devices

multisite/test: running the multisite test script with custom devices

3 years agoMerge pull request #45666 from SUSE/wip-fix-55087
Tim Serong [Tue, 29 Mar 2022 07:01:46 +0000 (18:01 +1100)]
Merge pull request #45666 from SUSE/wip-fix-55087

ceph.spec.in: Use libthrift-devel on SUSE distros

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #45664 from SUSE/wip-fix-55079
Tim Serong [Tue, 29 Mar 2022 06:08:09 +0000 (17:08 +1100)]
Merge pull request #45664 from SUSE/wip-fix-55079

ceph.spec.in: remove build directory at end of %install

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoceph.spec.in: use gcc-toolset-10 for building crimson 45676/head
Radosław Zarzyński [Mon, 28 Mar 2022 14:38:44 +0000 (16:38 +0200)]
ceph.spec.in: use gcc-toolset-10 for building crimson

This commit bumps up the toolset version but only to build crimson.
That is, the classical OSD stays unaffected.

The reason behind the upgrade is the following FTBFS:

```
[ 32%] Building CXX object src/seastar/CMakeFiles/seastar.dir/src/core/reactor.cc.o
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc: In constructor ‘seastar::reactor::reactor(std::shared_ptr<seastar::smp>, seastar::alien::instance&, unsigned int, seastar::reactor_backend_selector, seastar::reactor_config)’:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:926:90: error: use of deleted function ‘seastar::condition_variable::condition_variable()’
  926 |     , _thread_pool(std::make_unique<thread_pool>(this, seastar::format("syscall-{}", id))) {
      |                                                                                          ^
In file included from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/reactor.hh:74,
                 from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:32:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/condition-variable.hh:157:5: note: ‘seastar::condition_variable::condition_variable() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘’
  157 |     condition_variable() noexcept = default;
```

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
3 years agoMerge pull request #45647 from rhcs-dashboard/fix-55060-master 45682/head
Ernesto Puerta [Mon, 28 Mar 2022 15:29:11 +0000 (17:29 +0200)]
Merge pull request #45647 from rhcs-dashboard/fix-55060-master

mgr/dashboard: fix api test issue with pip

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: zmc <NOT@FOUND>
3 years agomgr/dashboard: Pool overall performance shows multiple entries of same pool in pool... 45335/head
Aashish Sharma [Thu, 10 Mar 2022 12:20:43 +0000 (17:50 +0530)]
mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview

This PR intends to fix this issue

Fixes:https://tracker.ceph.com/issues/54513
Signed-off-by: Aashish Sharma <aasharma@redhat.com>