]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoMerge pull request #44097 from cfsnyder/wip-53389-octopus
Yuri Weinstein [Fri, 10 Dec 2021 23:43:04 +0000 (15:43 -0800)]
Merge pull request #44097 from cfsnyder/wip-53389-octopus

octopus: osd/OSDMap.cc: clean up pg_temp for nonexistent pgs

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
3 years agoMerge pull request #44254 from neha-ojha/wip-perf-octopus
Yuri Weinstein [Fri, 10 Dec 2021 19:11:25 +0000 (11:11 -0800)]
Merge pull request #44254 from neha-ojha/wip-perf-octopus

octopus: qa: miscellaneous perf suite fixes

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoMerge pull request #43962 from cfsnyder/wip-53200-octopus
Yuri Weinstein [Fri, 10 Dec 2021 18:29:12 +0000 (10:29 -0800)]
Merge pull request #43962 from cfsnyder/wip-53200-octopus

octopus: osd: fix 'ceph osd stop <osd.nnn>' doesn't take effect

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43861 from pponnuvel/wip-53198-octopus
Yuri Weinstein [Fri, 10 Dec 2021 18:26:29 +0000 (10:26 -0800)]
Merge pull request #43861 from pponnuvel/wip-53198-octopus

octopus: mon/MgrStatMonitor: ignore MMgrReport from non-active mgr

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43887 from ifed01/wip-ifed-more-errors-shared-blob-repair-oct
Yuri Weinstein [Thu, 9 Dec 2021 21:29:14 +0000 (13:29 -0800)]
Merge pull request #43887 from ifed01/wip-ifed-more-errors-shared-blob-repair-oct

octopus: os/bluestore: fix additional errors during missed shared blob repair.

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43885 from ifed01/wip-ifed-fix-invalid-offset-repair-oct
Yuri Weinstein [Thu, 9 Dec 2021 21:28:43 +0000 (13:28 -0800)]
Merge pull request #43885 from ifed01/wip-ifed-fix-invalid-offset-repair-oct

octopus: os/bluestore: fix writing to invalid offset when repairing

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43883 from ifed01/wip-ifed-fix-53011-oct
Yuri Weinstein [Thu, 9 Dec 2021 21:28:06 +0000 (13:28 -0800)]
Merge pull request #43883 from ifed01/wip-ifed-fix-53011-oct

octopus: os/bluestore: use proper prefix when removing undecodable Share Blob.

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43757 from ifed01/wip-ifed-fix-write-small-head-pad-oct
Yuri Weinstein [Thu, 9 Dec 2021 21:27:22 +0000 (13:27 -0800)]
Merge pull request #43757 from ifed01/wip-ifed-fix-write-small-head-pad-oct

octopus: os/bluestore: _do_write_small fix head_pad

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44172 from cfsnyder/wip-52074-octopus
Yuri Weinstein [Thu, 9 Dec 2021 16:19:56 +0000 (08:19 -0800)]
Merge pull request #44172 from cfsnyder/wip-52074-octopus

octopus: rgw: user stats showing 0 value for "size_utilized" and "size_kb_utilized" fields

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44170 from cfsnyder/wip-53133-octopus
Yuri Weinstein [Thu, 9 Dec 2021 16:19:04 +0000 (08:19 -0800)]
Merge pull request #44170 from cfsnyder/wip-53133-octopus

octopus: rgw: disable prefetch in rgw_file to fix 3x read amplification

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoqa/suites/rados/perf/ceph.yaml: remove rgw 44254/head
Neha Ojha [Mon, 9 Aug 2021 14:35:01 +0000 (14:35 +0000)]
qa/suites/rados/perf/ceph.yaml: remove rgw

This is no longer required because we removed cosbench workloads in
fd350fd0150a2d4072f055658c20314a435a19ba. This is also required to prevent
failures like the following or any other changes that break the rgw task:

```
2021-08-06T20:13:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:curl: (7) Failed to connect to smithi060.front.sepia.ceph.com port 80: Connection refused
2021-08-06T20:15:33.813 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/contextutil.py", line 31, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_3c0f8c8164075af7aac4d1f2805d3f4580709461/qa/tasks/rgw.py", line 191, in start_rgw
    wait_for_radosgw(url, remote)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_3c0f8c8164075af7aac4d1f2805d3f4580709461/qa/tasks/util/rgw.py", line 94, in wait_for_radosgw
    assert exit_status == 0
AssertionError
```

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 119544bb29e253322af33e593ffd09e325c2af8a)

3 years agoqa: remove cosbench workloads from perf suites
Neha Ojha [Fri, 21 May 2021 20:17:11 +0000 (20:17 +0000)]
qa: remove cosbench workloads from perf suites

Due to https://tracker.ceph.com/issues/49139

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit fd350fd0150a2d4072f055658c20314a435a19ba)

3 years agoqa: use ubuntu_latest for perf suites
Neha Ojha [Fri, 21 May 2021 16:15:37 +0000 (16:15 +0000)]
qa: use ubuntu_latest for perf suites

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 5957d1797a4f67b4545c2554dff240463af87359)

3 years agoMerge pull request #44227 from ceph/octopus-mistune
Deepika Upadhyay [Wed, 8 Dec 2021 23:32:41 +0000 (05:02 +0530)]
Merge pull request #44227 from ceph/octopus-mistune

doc: Use older mistune

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43863 from ivancich/wip-multipart-purge-fix-octopus
Yuri Weinstein [Tue, 7 Dec 2021 18:56:26 +0000 (10:56 -0800)]
Merge pull request #43863 from ivancich/wip-multipart-purge-fix-octopus

octopus: rgw: fix bucket purge incomplete multipart uploads

Reviewed-by: Adam Emerson <aemerson@redhat.com>
3 years agoMerge pull request #43961 from cfsnyder/wip-53272-octopus
Yuri Weinstein [Tue, 7 Dec 2021 17:00:17 +0000 (09:00 -0800)]
Merge pull request #43961 from cfsnyder/wip-53272-octopus

octopus: rgw/beast: optimizations for request timeout

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43810 from cbodley/wip-qa-rgw-java-octopus
Yuri Weinstein [Tue, 7 Dec 2021 16:59:17 +0000 (08:59 -0800)]
Merge pull request #43810 from cbodley/wip-qa-rgw-java-octopus

qa/rgw: octopus branch targets ceph-octopus branch of java_s3tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
3 years agoMerge pull request #43696 from cfsnyder/wip-52959-octopus
Yuri Weinstein [Tue, 7 Dec 2021 16:58:19 +0000 (08:58 -0800)]
Merge pull request #43696 from cfsnyder/wip-52959-octopus

octopus: rgw/rgw_rados: make RGW request IDs non-deterministic

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agodoc: Use older mistune 44227/head
David Galloway [Mon, 6 Dec 2021 15:32:56 +0000 (10:32 -0500)]
doc: Use older mistune

https://github.com/miyakogi/m2r/issues/66

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit ed2ad24a4ba3ad3f8103926bfea2466b9eb61222)

3 years agorgw: user stats showing 0 value for "size_utilized" and "size_kb_utilized" fields 44172/head
J. Eric Ivancich [Wed, 28 Jul 2021 17:52:29 +0000 (13:52 -0400)]
rgw: user stats showing 0 value for "size_utilized" and "size_kb_utilized" fields

When accumulating user stats, the "utilized" fields are not looked
at. Updates RGWStorageStats::dump so it only outputs the "utilized"
data if they're updated.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 248fbce6b54f6c91e63b05861f8631ca64c8df81)

Conflicts:
src/rgw/rgw_admin.cc

Cherry-pick notes:
- Conflicts due to change of interface for rgw user and rados store

3 years agorgw: disable prefetch in rgw_file 44170/head
Kajetan Janiak [Wed, 18 Nov 2020 10:42:07 +0000 (11:42 +0100)]
rgw: disable prefetch in rgw_file

Each call to rgw_read (rgw_file.cc) invokes three calls to RGWRados::get_obj_state with s->prefetch_data=true. It results in great read amplification. If length argument in rgw_read call is smaller than rgw_max_chunk_size, then the amplification is threefold.

Signed-off-by: Kajetan Janiak <kjaniak@cloudferro.com>
(cherry picked from commit f915e21e5a1baf6030c1407b3058d4f58c638df9)

Conflicts:
src/rgw/rgw_op.cc

Cherry-pick notes:
- Octopus sets prfetch data flag through Rados method vs. method on object

3 years agoMerge pull request #42958 from ifed01/wip-ifed-fix-huge-omap-rename-oct
Igor Fedotov [Thu, 25 Nov 2021 10:23:17 +0000 (13:23 +0300)]
Merge pull request #42958 from ifed01/wip-ifed-fix-huge-omap-rename-oct

octopus: os/bluestore: cap omap naming scheme upgrade transactoin.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoosd/OSDMap.cc: clean up pg_temp for nonexistent pgs 44097/head
Cory Snyder [Wed, 17 Nov 2021 18:07:45 +0000 (13:07 -0500)]
osd/OSDMap.cc: clean up pg_temp for nonexistent pgs

Fixes an issue where the OSDMap does not clear pg-temp entries for PGs that no longer exist.

Fixes: https://tracker.ceph.com/issues/53308
Signed-off-by: Cory Snyder <csnyder@iland.com>
(cherry picked from commit 86367ea008281cf4398073466f3ece5ea18e82af)

3 years agoMerge pull request #43959 from guits/wip-53283-octopus
Guillaume Abrioux [Tue, 23 Nov 2021 16:43:56 +0000 (17:43 +0100)]
Merge pull request #43959 from guits/wip-53283-octopus

octopus: ceph-volume: `get_first_lv()` refactor

3 years agoos/bluestore: Fix omap upgrade to per-pg scheme 42958/head
Adam Kupczyk [Sat, 13 Nov 2021 10:28:18 +0000 (11:28 +0100)]
os/bluestore: Fix omap upgrade to per-pg scheme

This is fix to regression introduced by fix to omap upgrade: https://github.com/ceph/ceph/pull/43687
The problem was that we always skipped first omap entry.
This worked fine with objects having omap header key.
For objects without header key we skipped first actual omap key.

Fixes: https://tracker.ceph.com/issues/53260
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 65a3f374aa1c57c5bb9401e57dab98a643b4360a)

3 years agoceph-volume: `get_first_*()` refactor 43959/head
Guillaume Abrioux [Mon, 8 Mar 2021 08:59:26 +0000 (09:59 +0100)]
ceph-volume: `get_first_*()` refactor

As indicated by commit 17957d9beb42a04b8f180ccb7ba07d43179a41d3 those
fuctions were meant to avoid writing something like following:

```
lvs = get_lvs()
if len(lvs) >= 1:
lvs = lv[0]
```

Those functions should return `None` if 0 or more than 1 item is returned.
The current name of these functions are confusing and can lead to thinking that
we just want the first item returned, even though it returns more than 1
item, let's rename them to `get_single_pv()`, `get_single_vg()` and
`get_single_lv()`

Closes: https://tracker.ceph.com/issues/49643
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit a5e4216b49704783c55fb83b3ae6dde35b0082ad)

3 years agoMerge pull request #43952 from guits/wip-52597-octopus
Guillaume Abrioux [Thu, 18 Nov 2021 08:36:35 +0000 (09:36 +0100)]
Merge pull request #43952 from guits/wip-52597-octopus

octopus: ceph-volume: util/prepare fix osd_id_available()

3 years agoMerge pull request #43950 from guits/wip-53189-octopus
Guillaume Abrioux [Thu, 18 Nov 2021 08:35:32 +0000 (09:35 +0100)]
Merge pull request #43950 from guits/wip-53189-octopus

octopus: ceph-volume: fix a typo causing AttributeError

3 years agoosd: fix 'ceph osd stop <osd.nnn>' doesn't take effect 43962/head
tancz1 [Tue, 26 Oct 2021 09:25:20 +0000 (17:25 +0800)]
osd: fix 'ceph osd stop <osd.nnn>' doesn't take effect

when the osd state is in the non-active state, the osd daemon can be stopped.

Fixes: https://tracker.ceph.com/issues/53039
Signed-off-by: tan changzhi <544463199@qq.com>
(cherry picked from commit d595c95ef6c3dc34b8389ff4270639ff1550d269)

3 years agorgwi/beast: stream timer with duration 0 disables timeout 43961/head
Mark Kogan [Mon, 15 Nov 2021 15:50:49 +0000 (15:50 +0000)]
rgwi/beast: stream timer with duration 0 disables timeout

fixes all S3 operations failing with:
`2021-11-15T15:46:05.992+0000 7ffee17fa700 20 failed to read header: Bad file descriptor`
when `--rgw_frontends="beast port=8000 request_timeout_ms=0"`

Signed-off-by: Mark Kogan <mkogan@redhat.com>
3 years agorgw/beast: reference count Connections for timeout_handler
Casey Bodley [Thu, 11 Nov 2021 17:01:06 +0000 (12:01 -0500)]
rgw/beast: reference count Connections for timeout_handler

resolves a use-after-free in the timeout_handler, where a timeout fires
and schedules the timeout_handler for execution, but the coroutine exits
and destroys the socket before asio executes the timeout_handler

timeout_handler now holds a reference on the Connection to extend its
lifetime

now that the Connection is allocated on the heap, we can include the
parse_buffer in this memory instead of allocating it separately

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agorgw/beast: replace beast::tcp_stream with manual timeouts
Casey Bodley [Sat, 30 Oct 2021 23:47:02 +0000 (19:47 -0400)]
rgw/beast: replace beast::tcp_stream with manual timeouts

remove the beast::tcp_stream wrapper from the socket, and track timeouts
manually with a timeout_timer. this timer uses ceph's coarse_mono_clock
which is cheaper to sample than std::chrono::steady_clock

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agorgw/beast: use explicit executor type for tcp socket and stream
Casey Bodley [Mon, 1 Nov 2021 17:18:54 +0000 (13:18 -0400)]
rgw/beast: use explicit executor type for tcp socket and stream

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agospawn: use explicit strand executor
Casey Bodley [Mon, 1 Nov 2021 17:14:16 +0000 (13:14 -0400)]
spawn: use explicit strand executor

the default spawn::yield_context uses the polymorphic boost::asio::executor
to support any executor type

rgw's beast frontend always uses the same executor type for these
coroutines, so we can use that type directly to avoid the overhead of
type erasure and virtual function calls

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9d9258e06b78bb47fd0156d9bd7bb00b52a726b0)

Conflicts:
src/common/async/yield_context.h
src/rgw/rgw_d3n_cacherequest.h
src/rgw/rgw_notify.cc
src/rgw/rgw_sync_checkpoint.cc

Cherry-pick notes:
- src/rgw/rgw_d3n_cacherequest.h doesn't exist in Octopus
- src/rgw/rgw_sync_checkpoint.cc doesn't exist in Octopus
- conflicts due to rename of structs after Octopus
- conflicts due to macro for conditional inclusion of beast context in Octopus

3 years agorgw: clean up WITH_RADOSGW_BEAST_OPENSSL
Casey Bodley [Sat, 30 Oct 2021 23:45:45 +0000 (19:45 -0400)]
rgw: clean up WITH_RADOSGW_BEAST_OPENSSL

the #ifdef was covering more includes than it should have

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 44f4b083dea933f62cd76279593e3f1b3cd21f77)

3 years agoceph-volume: util/prepare fix osd_id_available() 43952/head
Guillaume Abrioux [Thu, 9 Sep 2021 08:23:43 +0000 (10:23 +0200)]
ceph-volume: util/prepare fix osd_id_available()

The current check only allows to request an OSD id that exists but
marked as 'destroyed'.
With this small fix, we can now use `--osd-id` with an id that doesn't
exist at all.

Fixes: https://tracker.ceph.com/issues/50880
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 73bfa5d2b0157f92721d8bf36619fd35ee265cdd)

3 years agoceph-volume: fix a typo causing AttributeError 43950/head
Taha Jahangir [Sun, 17 Oct 2021 05:00:27 +0000 (08:30 +0330)]
ceph-volume: fix a typo causing AttributeError

Signed-off-by: Taha Jahangir <mtjahangir@gmail.com>
(cherry picked from commit 4cdbba3344fe26b6351e88ce00a8655890a02115)

3 years agorgw: fix bucket purge incomplete multipart uploads 43863/head
J. Eric Ivancich [Mon, 8 Nov 2021 19:05:24 +0000 (14:05 -0500)]
rgw: fix bucket purge incomplete multipart uploads

The marker was not working correctly as segments of the bucket index
were listed to shut down any incomplete multipart uploads. This fixes
the marker, so it's maintained properly across iterations.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 9f2c2d901dff0acc16f80cb6ad32bb8c39c9ac6e)

Conflicts:
        src/rgw/rgw_multi.cc
        src/rgw/rgw_multi.h
 - dpp changes

3 years agoos/bluestore: fix additional errors during missed shared blob repair. 43887/head
Igor Fedotov [Wed, 21 Jul 2021 12:28:52 +0000 (15:28 +0300)]
os/bluestore: fix additional errors during missed shared blob repair.

Fixes: https://tracker.ceph.com/issues/51762
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d92ebd3ebea9a153c22a711bb2aae0ce17f5b304)

3 years agotest/objectstore/store_test: reveal incomplete fix for missed shared
Igor Fedotov [Wed, 21 Jul 2021 12:27:33 +0000 (15:27 +0300)]
test/objectstore/store_test: reveal incomplete fix for missed shared
blob repair.

Related-to: https://tracker.ceph.com/issues/51762
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 9893710328cef0942c86676e2dee72ee1fbecffd)

3 years agoos/bluestore: fix improper offset calculation when repairing. 43885/head
Igor Fedotov [Thu, 15 Jul 2021 12:10:14 +0000 (15:10 +0300)]
os/bluestore: fix improper offset calculation when repairing.

While repairing misreferenced blobs BlueStore could improperly calculate
an offset within a blob being fixed. This could happen when single
physical extent has been replaced by multiple ones - the following
pextent (if any in the current blob) would be treated with the improper offset within the blob. Offset calculation didn't account for each of that new pextents but the last one only.

Fixes: https://tracker.ceph.com/issues/51682
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ca4b6675fc3fd2f4cadad58044c97c5bb23d5938)

3 years agotest/objectstore/bluestore_types: add map_bl test case
Igor Fedotov [Thu, 15 Jul 2021 11:16:39 +0000 (14:16 +0300)]
test/objectstore/bluestore_types: add map_bl test case

Along with the basic bluestore_blob_t::map_any functionality
verification this UT shows how invalid offset might appear in
https://tracker.ceph.com/issues/51682

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 81f8e063c6f15d7763f51247babe2db7bf4c2aae)

3 years agoos/bluestore: use proper prefix when removing undecodable Share Blob. 43883/head
Igor Fedotov [Thu, 21 Oct 2021 15:39:54 +0000 (18:39 +0300)]
os/bluestore: use proper prefix when removing undecodable Share Blob.

Fixes: https://tracker.ceph.com/issues/53011
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit aaa0a172080a5a9ecba76be364af9a5277bc2187)

3 years agoMerge pull request #43747 from mfoliveira/wip-53100-octopus
Yuri Weinstein [Wed, 10 Nov 2021 23:35:16 +0000 (15:35 -0800)]
Merge pull request #43747 from mfoliveira/wip-53100-octopus

octopus: os/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_* options

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 years agoos/bluestore: fix invalid omap name conversion when upgrading to per-pg.
Igor Fedotov [Wed, 27 Oct 2021 10:59:34 +0000 (13:59 +0300)]
os/bluestore: fix invalid omap name conversion when upgrading to per-pg.

Fixes: https://tracker.ceph.com/issues/53062
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit cbc97018d883333f81ab9a3cfa99d2f68a9874cd)
(cherry picked from commit dc0a7e49434f76d97016934feed9a8ec806d1e42)

3 years agoMerge pull request #43658 from badone/wip-octopus-ceph-ansible-systemd-bug
Yuri Weinstein [Wed, 10 Nov 2021 16:28:25 +0000 (08:28 -0800)]
Merge pull request #43658 from badone/wip-octopus-ceph-ansible-systemd-bug

octopus: qa/ceph-ansible: Bump OS version for centos

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoos/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_search_bytes 43747/head
Kefu Chai [Tue, 1 Jun 2021 11:14:33 +0000 (19:14 +0800)]
os/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_search_bytes

so AvlAllocator can switch from the first-first mode to best-fit mode
without walking through the whole space map tree. in the
highly-fragmented system, iterating the whole tree could hurt the
performance of fast storage system a lot.

the idea comes from openzfs's metaslab allocator.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5a26875049d13130ffe5954428da0e1b9750359f)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
 Conflicts:
src/common/options/global.yaml.in:
- Moved new option into src/common/options.cc

3 years agoos/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_search_count
Kefu Chai [Wed, 2 Jun 2021 07:57:04 +0000 (15:57 +0800)]
os/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_search_count

so AvlAllocator can switch from the first-first mode to best-fit mode
without walking through the whole space map tree. in the
highly-fragmented system, iterating the whole tree could hurt the
performance of fast storage system a lot.

the idea comes from openzfs's metaslab allocator.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 40f05b971f5a8064cf9819f80fc3bbf21d5206da)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
 Conflicts:
src/common/options/global.yaml.in
- Moved new option into src/common/options.cc

3 years agoos/bluestore/AvlAllocator: use cbit for counting the order of alignment
Kefu Chai [Wed, 2 Jun 2021 08:31:18 +0000 (16:31 +0800)]
os/bluestore/AvlAllocator: use cbit for counting the order of alignment

no need to calculate the alignment first, cbits() would suffice. as it
counts the first set bit and the follow 0's in a number. the result
is identical to the cbit(alignment of that number).

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 573cbb796e8ba2f433caa308925735101a8161a6)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
3 years agoos/bluestore/AvlAllocator: use delegated ctor
Kefu Chai [Tue, 1 Jun 2021 10:52:11 +0000 (18:52 +0800)]
os/bluestore/AvlAllocator: use delegated ctor

less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9b52ba1dd0a5e199833d7ab2561a7b388d85afc1)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
 Conflicts:
src/os/bluestore/AvlAllocator.cc
        - Replace `std::string_view name` w/ `const std::string& name`.

3 years agoos/bluestore/AvlAllocator: specialize _block_picker()
Kefu Chai [Wed, 2 Jun 2021 07:11:30 +0000 (15:11 +0800)]
os/bluestore/AvlAllocator: specialize _block_picker()

before this change AvlAllocator::_block_picker() is used by both the
best-fit mode and first-fit mode. but since we cannot achieve the
locality by searching in the area pointed by curosr in best-fit mode,
we just pass a dummy cursor to AvlAllocator::_block_picker() when
searching in the best-fit mode.

but since the range_size_tree is already sorted by the size of ranges,
if _block_picker() fails to find one by the size, we should just give
up right away, and instead try again using a smaller size.

after this change, instead of sharing AvlAllocator::_block_picker()
across both the first-fit mode and the best-fit mode, this method
is specialize to two different variants: one for first-fit, and the
simplified one for best-fit.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4837166f9e7a659742d4184f021ad12260247888)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
3 years agoos/bluestore: Improve _block_picker function
Adam Kupczyk [Wed, 19 May 2021 10:49:37 +0000 (12:49 +0200)]
os/bluestore: Improve _block_picker function

Make _block_picker function scan (*cursor, end) + (begin, *cursor) instead of (*cursor, end) + (begin, end).
The second run over range (*cursor, end) could never yield any results.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit c732060d3e3ef96c6da06c9dde3ed8c064a50965)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
3 years agoos/bluestore: do not call _block_picker() again if already searched from start()
jhonxue [Wed, 18 Nov 2020 09:41:57 +0000 (17:41 +0800)]
os/bluestore: do not call _block_picker() again if already searched from start()

Fixes: https://tracker.ceph.com/issues/48272
Signed-off-by: Xue Yantao <jhonxue@tencent.com>
(cherry picked from commit fd5ca26e4a23d6c8992ab5927ce85ade958e251f)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
3 years agooctopus: qa/ceph-ansible: Bump OS version for centos 43658/head
Brad Hubbard [Mon, 25 Oct 2021 21:18:26 +0000 (07:18 +1000)]
octopus: qa/ceph-ansible: Bump OS version for centos

The systemd version in the 8.3 image is buggy so use 8.4 instead.

Fixes: https://tracker.ceph.com/issues/52923
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agomon/MgrStatMonitor: ignore MMgrReport from non-active mgr 43861/head
Sage Weil [Thu, 18 Mar 2021 16:45:48 +0000 (11:45 -0500)]
mon/MgrStatMonitor: ignore MMgrReport from non-active mgr

If it's not the active mgr, we should ignore it.

Since the mgr instance is best identified by the gid, add that to the
message.  (We can't use the source_addrs for the message since that is
the MgrStandby monc addr, not the active mgr addrs in the MgrMap.)

This fixes a problem where a just-demoted mgr report gets processed and a
new mgr gets a ServiceMap with an epoch >= its pending map.  (At least,
that is my theory!)

Fixes: https://tracker.ceph.com/issues/48022
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4d447092c3542bf57dfb4942db766adf2923c069)

Conflicts:
src/messages/MMonMgrReport.h
src/mon/MgrStatMonitor.cc

3 years agomgr: tell monc when we get new servicemap, fsmap
Sage Weil [Thu, 18 Mar 2021 15:40:42 +0000 (10:40 -0500)]
mgr: tell monc when we get new servicemap, fsmap

Otherwise, when we re-subscribe we'll request an old map again.  In the
case of the servicemap, that can lead to a failed assertion.

Fixes: https://tracker.ceph.com/issues/48022
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3dbc1f0578d944217ad2bacb58ef561e678abb6c)

3 years agoos/bluestore: cap omap naming scheme upgrade transactoin.
Igor Fedotov [Tue, 9 Feb 2021 15:29:01 +0000 (18:29 +0300)]
os/bluestore: cap omap naming scheme upgrade transactoin.

We shouldn't use single per-onode transaction for such an upgrade when onode's omap list is huge. This results in similarly sized WAL/SST files which are inefficient, might cause high memory usage and sometimes error-prone.

Fixes: https://tracker.ceph.com/issues/49170
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e897fa243c1dd38329733b452872616023f14ac8)

 Conflicts (caused by lack of per-pg omap scheme):
src/os/bluestore/BlueStore.cc
src/os/bluestore/BlueStore.h
src/os/bluestore/bluestore_types.h

3 years agoqa/rgw: octopus branch targets ceph-octopus branch of java_s3tests 43810/head
Casey Bodley [Thu, 4 Nov 2021 16:12:04 +0000 (12:12 -0400)]
qa/rgw: octopus branch targets ceph-octopus branch of java_s3tests

this commit is applied directly to the octopus branch instead of a
backport from master, because it targets the ceph-octopus branch instead
of the ceph-master branch on master

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agorgw/rgw_rados: make RGW request IDs non-deterministic 43696/head
Cory Snyder [Tue, 5 Oct 2021 19:07:11 +0000 (19:07 +0000)]
rgw/rgw_rados: make RGW request IDs non-deterministic

Use a random number vs. incremental counter for first component of request ID.

Fixes: https://tracker.ceph.com/issues/52818
Signed-off-by: Cory Snyder <csnyder@iland.com>
(cherry picked from commit bce34dd68634d241b451111dcf2e931837eb4bfd)

3 years agoMerge pull request #43758 from tchaikov/octopus-pr-43748
Sebastian Wagner [Tue, 2 Nov 2021 08:14:55 +0000 (09:14 +0100)]
Merge pull request #43758 from tchaikov/octopus-pr-43748

octopus: admin/doc-requirements.txt: pin Sphinx at 3.5.4

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agotest/rgw: use spawn library for test_rgw_dmclock_scheduler 43758/head
Casey Bodley [Tue, 20 Jul 2021 16:50:25 +0000 (12:50 -0400)]
test/rgw: use spawn library for test_rgw_dmclock_scheduler

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a8e3589a2c875b6fadc853c75f20cb9256f294ca)

Conflicts:
src/test/rgw/CMakeLists.txt
src/test/rgw/test_rgw_dmclock_scheduler.cc: trivial resolution

3 years agotest/rgw: fix use of poll() with timers in unittest_rgw_dmclock_scheduler
Casey Bodley [Mon, 19 Jul 2021 22:07:47 +0000 (18:07 -0400)]
test/rgw: fix use of poll() with timers in unittest_rgw_dmclock_scheduler

the AsyncScheduler uses an asio timer to dispatch work to its executor
with an optional delay. when no delay is requested, it waits on the
timer with an expiration time in the past (crimson::dmclock::TimeZero)

tests are failing here because poll() is returning without executing the
handlers of those expired timers

asio implements these timers with timerfd and epoll. debugging with
strace, i see that these timers armed with timerfd_settime() are not
always immediately ready according to epoll_wait():

  eventfd2(0, EFD_CLOEXEC|EFD_NONBLOCK)   = 3
  epoll_create1(EPOLL_CLOEXEC)            = 4
  timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC) = 5
  epoll_ctl(4, EPOLL_CTL_ADD, 3, {events=EPOLLIN|EPOLLERR|EPOLLET, data={u32=14164052, u64=14164052}}) = 0
  epoll_ctl(4, EPOLL_CTL_ADD, 5, {events=EPOLLIN|EPOLLERR, data={u32=14164064, u64=14164064}}) = 0
  timerfd_settime(5, TFD_TIMER_ABSTIME, {it_interval={tv_sec=0, tv_nsec=0}, it_value={tv_sec=0, tv_nsec=1}}, {it_interval={tv_sec=0, tv_nsec=0}, it_value={tv_sec=0, tv_nsec=0}}) = 0
  epoll_wait(4, [{events=EPOLLIN, data={u32=14164052, u64=14164052}}], 128, 0) = 1
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [], 128, 0)               = 0
  epoll_wait(4, [{events=EPOLLIN, data={u32=14164064, u64=14164064}}], 128, 0) = 1

in this example, it took 6 calls to context.poll() before it was ready
to execute the timer's handler

to work around this, replace calls to context.poll() with calls to
context.run_for() with a very short duration

Fixes: https://tracker.ceph.com/issues/42788
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 21baed999e31c5e69c75f0cbb8757ef91585d917)

3 years agoinclude/rados/librados.h: avoid redefinition of rados_object_list_item
Kefu Chai [Fri, 28 Aug 2020 11:25:51 +0000 (19:25 +0800)]
include/rados/librados.h: avoid redefinition of rados_object_list_item

doxygen complains at seeing rados_object_list_item gets defined twice.
so let's fix it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c53a02ac56e4e87631262fffc711e2e009561d7)

3 years agomgr/dashboard: pin a version for autopep8 and pyfakefs
Nizamudeen A [Mon, 25 Oct 2021 08:42:57 +0000 (14:12 +0530)]
mgr/dashboard: pin a version for autopep8 and pyfakefs

Fixes: https://tracker.ceph.com/issues/53024
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 946dab4f608ec47e0a3cfefdf8e7d1afda69117f)

Conflicts:
src/pybind/mgr/dashboard/requirements-lint.txt: trivial
resolution

3 years agoadmin/doc-requirements.txt: pin Sphinx at 3.5.4
Kefu Chai [Sat, 30 Oct 2021 03:18:17 +0000 (11:18 +0800)]
admin/doc-requirements.txt: pin Sphinx at 3.5.4

* pin Sphinx at 3.5.4
* pin docutils at 0.18

at least the combination of these two versions
is known to compile.

to address the bug reported at
https://sourceforge.net/p/docutils/bugs/431/

the backtrace looks like:

/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/sphinx/util/docutils.py:285:
RemovedInSphinx30Warning: function based directive support is now
deprecated. Use class based directive instead.
  warnings.warn('function based directive support is now deprecated. '

Exception occurred:
  File
"/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/docutils/writers/html5_polyglot/__init__.py",
line 445, in section_title_tags
    if (ids and self.settings.section_self_link
AttributeError: 'Values' object has no attribute 'section_self_link'

please note this change is not cherry-picked from
master, because master already bumped Sphinx to 3.5.4
in 4968baa2523bd2a5ca6be147b26bc28906a864c9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 6362b554e2e57227c9fda51c7703f14674da8358)

Conflicts:
admin/doc-requirements.txt: trivial resolution

3 years agoadmin/doc-requirements.txt: require breathe >= 4.20
Kefu Chai [Mon, 14 Dec 2020 08:25:07 +0000 (16:25 +0800)]
admin/doc-requirements.txt: require breathe >= 4.20

to be compatible with Sphinx 3.2
see https://github.com/michaeljones/breathe/tree/v4.20.0#change-log

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2f1f16ed865114faedc3bdb3049da722284fab61)

3 years agodoc/scripts/gen_state_diagram.py: Fix literal comparison syntax warnings
Varsha Rao [Thu, 11 Jun 2020 06:36:38 +0000 (12:06 +0530)]
doc/scripts/gen_state_diagram.py: Fix literal comparison syntax warnings

In python3.8 comparing strings using 'is' and 'is not' throws syntax warning.
Instead use equality operator.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 61e7bcded852e90e6249ab0f3c37ec2688537c83)

3 years agodoc/conf.py: define CEPH_RADOS_API for breathe
Kefu Chai [Fri, 28 Aug 2020 10:20:13 +0000 (18:20 +0800)]
doc/conf.py: define CEPH_RADOS_API for breathe

otherwise we could have following errors:

Invalid C declaration: Expected identifier in nested name, got keyword: int [error at 18]
      CEPH_RADOS_API int rados_aio_append (rados_ioctx_t io, const char *oid, rados_completion_t completion, const char *buf, size_t len)
      ------------------^

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cf357e17ab5cbd9af4f67c51117b5f368cb9913f)

3 years agoMerge pull request #43441 from rhcs-dashboard/wip-52836-octopus
Yuri Weinstein [Mon, 1 Nov 2021 15:46:22 +0000 (08:46 -0700)]
Merge pull request #43441 from rhcs-dashboard/wip-52836-octopus

octopus: qa/mgr/dashboard/test_pool: don't check HEALTH_OK

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoos/bluestore: _do_write_small fix head_pad 43757/head
dheart [Tue, 12 Oct 2021 08:46:34 +0000 (16:46 +0800)]
os/bluestore: _do_write_small fix head_pad

Signed-off-by: dheart <dheart_joe@163.com>
(cherry picked from commit ed8dd300a88173b1e5efafb6bb061a15ea296c29)

3 years agoMerge pull request #43381 from nkshirsagar/octopus
Yuri Weinstein [Thu, 28 Oct 2021 21:53:01 +0000 (14:53 -0700)]
Merge pull request #43381 from nkshirsagar/octopus

octopus: rgw: clear buckets before calling list_buckets()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Prashant D <pdhange@redhat.com>
Reviewed-by: Ponnuvel Palaniyappan <pponnuvel@gmail.com>
3 years agoMerge pull request #43325 from gerald-yang/octopus-50482
Yuri Weinstein [Tue, 26 Oct 2021 20:33:15 +0000 (13:33 -0700)]
Merge pull request #43325 from gerald-yang/octopus-50482

octopus: msgr/async: fix unsafe access in unregister_conn()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43310 from gerald-yang/octopus-51199
Yuri Weinstein [Tue, 26 Oct 2021 20:31:35 +0000 (13:31 -0700)]
Merge pull request #43310 from gerald-yang/octopus-51199

octopus: msg/async: allow connection reaping to be tuned; fix cephfs test

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43557 from badone/wip-octopus-ceph-ansible-max-usable-version
Yuri Weinstein [Thu, 21 Oct 2021 20:27:47 +0000 (13:27 -0700)]
Merge pull request #43557 from badone/wip-octopus-ceph-ansible-max-usable-version

octopus: qa/ceph-ansible: Pin to last compatible stable release

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoMerge pull request #43607 from rhcs-dashboard/wip-52986-octopus
Ernesto Puerta [Thu, 21 Oct 2021 08:55:25 +0000 (10:55 +0200)]
Merge pull request #43607 from rhcs-dashboard/wip-52986-octopus

octopus: mgr/dashboard/api: set a UTF-8 locale when running pip

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agomgr/dashboard/api: set a UTF-8 locale when running pip 43607/head
Kefu Chai [Tue, 17 Aug 2021 07:53:51 +0000 (15:53 +0800)]
mgr/dashboard/api: set a UTF-8 locale when running pip

ansible-core started to include files whose filenames are encoded in
non-ascii characters, so we have to use a more capable encoding for the
locale in order to install this package. otherwise we'd have following
error:

Collecting ansible-core<2.12,>=2.11.3
  Using cached ansible-core-2.11.4.tar.gz (6.8 MB)
ERROR: Exception:

Traceback (most recent call last):
  File "/tmp/tmp.fX76ASIrch/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
...
  File "/tmp/tmp.fX76ASIrch/venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 226, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 05e4145856bb5ed19ecc879f2e50b5a88cb2045e)

3 years ago15.2.15 v15.2.15
Jenkins Build Slave User [Wed, 20 Oct 2021 14:19:57 +0000 (14:19 +0000)]
15.2.15

3 years agooctopus: qa/ceph-ansible: Pin to last compatible stable release 43557/head
Brad Hubbard [Mon, 11 Oct 2021 21:38:32 +0000 (07:38 +1000)]
octopus: qa/ceph-ansible: Pin to last compatible stable release

https://github.com/ceph/ceph-ansible/pull/6892 introduces a breaking
change so pin the CA version to stable-6.0 and ansible 2.9.

Fixes: https://tracker.ceph.com/issues/52943
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
3 years agorgw: clear buckets before calling list_buckets() 43381/head
Nikhil Kshirsagar [Fri, 1 Oct 2021 04:49:00 +0000 (10:19 +0530)]
rgw: clear buckets before calling list_buckets()

The ragodgw-admin bucket limit check command has a bug in
octopus. Since we do not clear the bucket list before
list_buckets() returns the next max_entries, they are appended
to the existing list and we end up counting the first ones again.

This bug is triggered if bucket count exceeds max_entries and
causes duplicates in the output of radosgw-admin bucket limit check.

The fix clears the buckets structure before the list_buckets()
populates it again with the next lot of buckets to iterate through.

partial manual cherry-pick of 99f7c4aa1286edfea6961b92bb44bb8fe22bd599

Signed-off-by: Nikhil Kshirsagar <nkshirsagar@gmail.com>
3 years agoMerge pull request #43418 from trociny/wip-51645-octopus
Yuri Weinstein [Thu, 7 Oct 2021 14:27:05 +0000 (07:27 -0700)]
Merge pull request #43418 from trociny/wip-51645-octopus

octopus: osd/OSD: mkfs need wait for transcation completely finish

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43407 from trociny/wip-52809-octopus
Yuri Weinstein [Thu, 7 Oct 2021 14:22:11 +0000 (07:22 -0700)]
Merge pull request #43407 from trociny/wip-52809-octopus

octopus: tools/erasure-code: new tool to encode/decode files

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43268 from cfsnyder/wip-52587-octopus
Guillaume Abrioux [Thu, 7 Oct 2021 08:31:08 +0000 (10:31 +0200)]
Merge pull request #43268 from cfsnyder/wip-52587-octopus

octopus: ceph-volume: fix lvm activate --all --no-systemd

3 years agoqa/mgr/dashboard/test_pool: don't check HEALTH_OK 43441/head
Ernesto Puerta [Wed, 22 Sep 2021 12:25:44 +0000 (14:25 +0200)]
qa/mgr/dashboard/test_pool: don't check HEALTH_OK

Fixes: https://tracker.ceph.com/issues/48845
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 2283cb068b82033b14587c7bac6a28440221dcd8)

3 years agoMerge pull request #43424 from cfsnyder/wip-51695-octopus
Yuri Weinstein [Wed, 6 Oct 2021 16:59:10 +0000 (09:59 -0700)]
Merge pull request #43424 from cfsnyder/wip-51695-octopus

octopus: rgw: fail as expected when set/delete-bucket-website attempted on a non-exis…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43314 from idryomov/wip-rbd-mirror-snapshot-rx-only-octopus
Yuri Weinstein [Tue, 5 Oct 2021 14:57:36 +0000 (07:57 -0700)]
Merge pull request #43314 from idryomov/wip-rbd-mirror-snapshot-rx-only-octopus

octopus: rbd-mirror: unbreak one-way snapshot-based mirroring

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
3 years agoMerge pull request #43312 from idryomov/wip-keyring-resolve-error-octopus
Yuri Weinstein [Tue, 5 Oct 2021 14:57:07 +0000 (07:57 -0700)]
Merge pull request #43312 from idryomov/wip-keyring-resolve-error-octopus

octopus: auth,mon: don't log "unable to find a keyring" error when key is given

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43263 from cfsnyder/wip-51552-octopus
Yuri Weinstein [Tue, 5 Oct 2021 14:56:28 +0000 (07:56 -0700)]
Merge pull request #43263 from cfsnyder/wip-51552-octopus

octopus: ceph-monstore-tool: use a large enough paxos/{first,last}_committed

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #42986 from MrFreezeex/wip-52460-octopus
Yuri Weinstein [Tue, 5 Oct 2021 14:55:24 +0000 (07:55 -0700)]
Merge pull request #42986 from MrFreezeex/wip-52460-octopus

octopus: rbd-mirror: add perf counters to snapshot replayed

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
3 years agoMerge pull request #42837 from steveftaylor/48212
Yuri Weinstein [Tue, 5 Oct 2021 14:46:28 +0000 (07:46 -0700)]
Merge pull request #42837 from steveftaylor/48212

octopus: mon/OSDMonitor: account for PG merging in epoch_by_pg accounting

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agorgw: fail as expected when set/delete-bucket-website attempted on a non-existent... 43424/head
mengxiangrui [Tue, 6 Jul 2021 11:58:55 +0000 (19:58 +0800)]
rgw: fail as expected when set/delete-bucket-website attempted on a non-existent bucket, rgw should return HTTP 404 and NoSuchBucket.

Fixes: https://tracker.ceph.com/issues/51536
Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
(cherry picked from commit c623aa45d35b269c6701a57e44ac05bb29a79dc8)

Conflicts:
- src/rgw/rgw_op.cc

Cherry-pick notes:
- rgw_op.cc forward_reqeuest_to_master takes different arguments in Octopus vs. Quincy

3 years agoosd/OSD: mkfs need wait for transcation completely finish 43418/head
Chen Fan [Wed, 9 Jun 2021 05:29:03 +0000 (13:29 +0800)]
osd/OSD: mkfs need wait for transcation completely finish

when do ceph-osd mkfs, when ceph-osd process exit, sometimes
the block data could be written incompletely. we need add
wait for it complete.

Signed-off-by: Chen Fan <fan.chen@easystack.cn>
(cherry picked from commit 0ffadad3a83b3ca634d7d58a80c84d1d8761e2ea)

3 years agoMerge pull request #43349 from cfsnyder/wip-52351-octopus
Yuri Weinstein [Mon, 4 Oct 2021 19:32:06 +0000 (12:32 -0700)]
Merge pull request #43349 from cfsnyder/wip-52351-octopus

octopus: rgw: fix sts memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43270 from cfsnyder/wip-51778-octopus
Daniel Gryniewicz [Mon, 4 Oct 2021 16:59:23 +0000 (12:59 -0400)]
Merge pull request #43270 from cfsnyder/wip-51778-octopus

octopus: rgw : add check for tenant provided in RGWCreateRole

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #43265 from cfsnyder/wip-52331-octopus
Yuri Weinstein [Mon, 4 Oct 2021 15:26:05 +0000 (08:26 -0700)]
Merge pull request #43265 from cfsnyder/wip-52331-octopus

octopus: cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43234 from MrFreezeex/wip-51838-octopus
Yuri Weinstein [Mon, 4 Oct 2021 15:19:57 +0000 (08:19 -0700)]
Merge pull request #43234 from MrFreezeex/wip-51838-octopus

octopus: ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43369 from tchaikov/octopus-pr-39602
Yuri Weinstein [Mon, 4 Oct 2021 15:14:10 +0000 (08:14 -0700)]
Merge pull request #43369 from tchaikov/octopus-pr-39602

octopus: mgr/influx: use "N/A" for unknown hostname

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43352 from rhcs-dashboard/wip-52773-octopus
Yuri Weinstein [Mon, 4 Oct 2021 15:13:06 +0000 (08:13 -0700)]
Merge pull request #43352 from rhcs-dashboard/wip-52773-octopus

octopus: qa/mgr/dashboard: add extra wait to test

Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agotest/erasure-code: remove ceph_erasure_code 43407/head
Mykola Golub [Wed, 25 Mar 2020 08:44:18 +0000 (08:44 +0000)]
test/erasure-code: remove ceph_erasure_code

Its functionality is moved to ceph-erasure-code-tool.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 56aaaf8a97574a0284bad37cc0ba5e1c262f33e0)

3 years agorpm,deb: add ceph-erasure-code-tool to ceph-osd package
Mykola Golub [Wed, 25 Mar 2020 08:37:00 +0000 (08:37 +0000)]
rpm,deb: add ceph-erasure-code-tool to ceph-osd package

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 53c75eebbdc9d85b07dd90608b45ad794a54f848)

3 years agoceph-erasure-code-tool: new tool to encode/decode files
Mykola Golub [Fri, 20 Mar 2020 16:19:25 +0000 (16:19 +0000)]
ceph-erasure-code-tool: new tool to encode/decode files

E.g. it may be useful as a last resort when recovering an object from
a damaged PG: extract the encoded object chunks from the PG shards
with ceph-objectstore-tool and then decode with ceph-erasure-code-tool.

It also has functionality similar to what ceph_erasure_code test provides.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit b274357d29fb25a29f75c08283fec185902e7970)