]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
19 months agorgw: initialize RGWIndexCompletionManager thread after related members 49093/head
Casey Bodley [Mon, 7 Mar 2022 20:10:07 +0000 (15:10 -0500)]
rgw: initialize RGWIndexCompletionManager thread after related members

resolves valgrind issues about RGWIndexCompletionManager::process()
using uninitialized memory

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 112c2608750cf8db2c52c8e89f5897080501f6bd)

19 months agorgw: prevent spurious/lost notifications in the index completion thread
Yuval Lifshitz [Wed, 23 Feb 2022 15:21:10 +0000 (17:21 +0200)]
rgw: prevent spurious/lost notifications in the index completion thread

this was happening when asyn completions happened during reshard.
more information about testing:
https://gist.github.com/yuvalif/d526c0a3a4c5b245b9e951a6c5a10517

we also add more logs to the completion manager.
should allow finding unhandled completions due to reshards.

Fixes: https://tracker.ceph.com/issues/54435
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit e72b6510a402cbafa6c99475626f15d97fd00f86)

19 months agoMerge pull request #54517 from lxbsz/wip-63513
Yuri Weinstein [Tue, 5 Dec 2023 19:10:15 +0000 (11:10 -0800)]
Merge pull request #54517 from lxbsz/wip-63513

pacific: mds: fix issuing redundant reintegrate/migrate_stray requests

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #50533 from k0ste/wip-55613-pacific
Yuri Weinstein [Tue, 5 Dec 2023 19:09:18 +0000 (11:09 -0800)]
Merge pull request #50533 from k0ste/wip-55613-pacific

pacific: RGW - Fix NoSuchTagSet error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
19 months agoMerge pull request #44476 from pritha-srivastava/wip-52784-pacific
Yuri Weinstein [Tue, 5 Dec 2023 19:08:40 +0000 (11:08 -0800)]
Merge pull request #44476 from pritha-srivastava/wip-52784-pacific

pacific: rgw/sts: createbucket op should take session_policies into account

Reviewed-by: Casey Bodley <cbodley@redhat.com>
19 months agoMerge pull request #52763 from dparmar18/wip-61732
Yuri Weinstein [Tue, 5 Dec 2023 16:37:59 +0000 (08:37 -0800)]
Merge pull request #52763 from dparmar18/wip-61732

pacific: qa: pass arg as list to fix test case failure

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #54713 from mchangir/wip-63588-pacific
Yuri Weinstein [Tue, 5 Dec 2023 16:20:40 +0000 (08:20 -0800)]
Merge pull request #54713 from mchangir/wip-63588-pacific

pacific: qa: run kernel_untar_build with newer tarball

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #54472 from lxbsz/wip-63512
Yuri Weinstein [Tue, 5 Dec 2023 16:18:23 +0000 (08:18 -0800)]
Merge pull request #54472 from lxbsz/wip-63512

pacific: client: queue a delay cap flushing if there are ditry caps/snapcaps

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #54033 from vshankar/wip-63173-pacific
Yuri Weinstein [Tue, 5 Dec 2023 16:17:40 +0000 (08:17 -0800)]
Merge pull request #54033 from vshankar/wip-63173-pacific

pacific: mds: adjust pre_segments_size for MDLog when trimming segments for st…

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #53981 from lxbsz/wip-62916
Yuri Weinstein [Tue, 5 Dec 2023 16:16:40 +0000 (08:16 -0800)]
Merge pull request #53981 from lxbsz/wip-62916

pacific: client: fix sync fs to force flush mdlog for all sessions

Reviewed-by: Kotresh HR khiremat@redhat.com
Reviewed-by: Venky Shankar <vshankar@redhat.com>
19 months agoMerge pull request #53574 from kotreshhr/wip-62406-pacific
Yuri Weinstein [Tue, 5 Dec 2023 16:14:46 +0000 (08:14 -0800)]
Merge pull request #53574 from kotreshhr/wip-62406-pacific

pacific:    mgr/volumes: Fix pending_subvolume_deletions in volume info

Reviewed-by: Kotresh HR khiremat@redhat.com
19 months agoMerge pull request #52987 from leonid-s-usov/backport/strip-nofail/pacific
Rishabh Dave [Tue, 5 Dec 2023 06:34:10 +0000 (12:04 +0530)]
Merge pull request #52987 from leonid-s-usov/backport/strip-nofail/pacific

pacific: nofail option in fstab not supported

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agomds: fix issuing redundant reintegrate/migrate_stray requests 54517/head
Xiubo Li [Fri, 15 Sep 2023 00:41:35 +0000 (08:41 +0800)]
mds: fix issuing redundant reintegrate/migrate_stray requests

Just in case a CInode's nlink is 1, and then a unlink request comes
and then early replies and submits to the MDLogs, but just before
the MDlogs are flushed a link request comes, and the link request
also succeeds and early replies to client.

Later when the unlink/link requests' MDLog events are flushed and
the callbacks are called, which will fire a stray denty reintegration.
But it will pick the new dentry, which is from the link's request
and is a remote dentry, to do the reintegration. While in the
'rename' code when traversing the path it will trigger to call the
'dn->link_remote()', which later will fire a new stray dentry
reintegration.

The problem is if the first 'rename' request is retried several
times, and in each time it will fire a new reintegration, which
makes no sense and maybe blocked for a very long time dues to some
reasons and then will be reported as slow request warning.

Fixes: https://tracker.ceph.com/issues/62702
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 53d9e657e42bdf2440cd93a8273eb3ce79d8bd33)

19 months agomds: record the internal client request and receive client reply
Xiubo Li [Fri, 15 Sep 2023 00:41:35 +0000 (08:41 +0800)]
mds: record the internal client request and receive client reply

This will be used to avoid possible multiple reintegration issue
later.

Fixes: https://tracker.ceph.com/issues/62702
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 723c1b7c9e773a68b6c2586155092ca95ab4f3e7)

Conflicts: with commit 7243b680526 ("mds: ensure next replay is
queued on req drop")

19 months agoMerge pull request #54314 from batrick/wip-63419-pacific
Yuri Weinstein [Mon, 4 Dec 2023 19:35:48 +0000 (11:35 -0800)]
Merge pull request #54314 from batrick/wip-63419-pacific

pacific: mds: ensure next replay is queued on req drop

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #54593 from idryomov/wip-53593-pacific
Yuri Weinstein [Mon, 4 Dec 2023 15:54:31 +0000 (07:54 -0800)]
Merge pull request #54593 from idryomov/wip-53593-pacific

pacific: osd: don't require RWEXCL lock for stat+write ops.

Reviewed-by: Laura Flores <lflores@redhat.com>
19 months agoMerge pull request #52993 from k0ste/wip-57474-pacific
Yuri Weinstein [Mon, 4 Dec 2023 15:53:28 +0000 (07:53 -0800)]
Merge pull request #52993 from k0ste/wip-57474-pacific

pacific: mgr: fix a race condition in DaemonServer::handle_report()

Reviewed-by: Laura Flores <lflores@redhat.com>
19 months agoMerge pull request #51259 from k0ste/wip-52557-pacific
Yuri Weinstein [Mon, 4 Dec 2023 15:52:40 +0000 (07:52 -0800)]
Merge pull request #51259 from k0ste/wip-52557-pacific

pacific: pybind/rados: don't close watch in dealloc if already closed

Reviewed-by: Laura Flores <lflores@redhat.com>
19 months agoMerge pull request #54039 from vshankar/wip-61829-pacific
Yuri Weinstein [Fri, 1 Dec 2023 17:22:36 +0000 (09:22 -0800)]
Merge pull request #54039 from vshankar/wip-61829-pacific

pacific: qa: assign file system affinity for replaced MDS

Reviewed-by: Laura Flores <lflores@redhat.com>
19 months agoMerge pull request #54361 from ifed01/wip-ifed-fix-rados-pybind-pac
Yuri Weinstein [Fri, 1 Dec 2023 16:41:28 +0000 (08:41 -0800)]
Merge pull request #54361 from ifed01/wip-ifed-fix-rados-pybind-pac

pacific: pybind/rados: fix missed changes for PEP484 style type annotations

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
19 months agoMerge pull request #51260 from k0ste/wip-52307-pacific
Yuri Weinstein [Fri, 1 Dec 2023 16:39:20 +0000 (08:39 -0800)]
Merge pull request #51260 from k0ste/wip-52307-pacific

pacific: doc: clarify use of `rados rm` command

Reviewed-by: Neha Ojha <nojha@redhat.com>
19 months agoMerge pull request #54652 from k0ste/wip-58829-pacific
Nizamudeen A [Wed, 29 Nov 2023 11:32:47 +0000 (17:02 +0530)]
Merge pull request #54652 from k0ste/wip-58829-pacific

pacific: mgr/dashboard: fix constraints.txt

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
19 months agoqa: run kernel_untar_build with newer tarball 54713/head
Milind Changire [Tue, 14 Nov 2023 10:05:50 +0000 (15:35 +0530)]
qa: run kernel_untar_build with newer tarball

Replace old tarball link with a url link to linux-6.5.11.tar.xz.

Fixes: https://tracker.ceph.com/issues/57655
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit b150e935e8433ce594d9ab25b16094e616d2f489)

19 months agorgw/sts: createbucket op should take into account 44476/head
Pritha Srivastava [Thu, 8 Jul 2021 15:54:10 +0000 (21:24 +0530)]
rgw/sts: createbucket op should take into account
session policies also while evaluating permissions.

Fixes: https://tracker.ceph.com/issues/51598
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 261eb60e0f3df202d0d13c719338690fbd6edb70)

19 months agoMerge pull request #54474 from k0ste/wip-63441-pacific
Yuri Weinstein [Tue, 28 Nov 2023 20:03:08 +0000 (12:03 -0800)]
Merge pull request #54474 from k0ste/wip-63441-pacific

pacific: Fixing example of BlueStore resharding.

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Joshua Baergen <jbaergen@digitalocean.com>
19 months agoMerge pull request #53567 from pdvian/wip-59178-pacific
Yuri Weinstein [Tue, 28 Nov 2023 20:00:59 +0000 (12:00 -0800)]
Merge pull request #53567 from pdvian/wip-59178-pacific

pacific: blk/kernel: Add O_EXCL for block devices

Reviewed-by: Laura Flores <lflores@redhat.com>
19 months agoMerge pull request #52055 from trociny/wip-61433-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:59:02 +0000 (11:59 -0800)]
Merge pull request #52055 from trociny/wip-61433-pacific

pacific: rgw: multisite data log flag not used

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #52051 from trociny/wip-59610-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:58:29 +0000 (11:58 -0800)]
Merge pull request #52051 from trociny/wip-59610-pacific

pacific: rgw/sts: AssumeRole no longer writes to user metadata

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #51602 from alimaredia/wip-58584-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:57:54 +0000 (11:57 -0800)]
Merge pull request #51602 from alimaredia/wip-58584-pacific

pacific: rgw: set keys from from master zone on admin api user create

Reviewed-by: Jiffin Tony Thottan <jthottan@redhat.com>
19 months agoMerge pull request #51600 from alimaredia/wip-61175-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:55:27 +0000 (11:55 -0800)]
Merge pull request #51600 from alimaredia/wip-61175-pacific

pacific: rgw/swift: check position of first slash in slo manifest files

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #51445 from k0ste/wip-59729-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:54:59 +0000 (11:54 -0800)]
Merge pull request #51445 from k0ste/wip-59729-pacific

pacific: RGW: Solving the issue of not populating etag in Multipart upload result

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #51256 from k0ste/wip-55063-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:54:25 +0000 (11:54 -0800)]
Merge pull request #51256 from k0ste/wip-55063-pacific

pacific: radosgw-admin: fix segfault on pipe modify without source/dest zone specified

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
19 months agoMerge pull request #50540 from k0ste/wip-55149-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:53:31 +0000 (11:53 -0800)]
Merge pull request #50540 from k0ste/wip-55149-pacific

pacific: rgw: Update "CEPH_RGW_DIR_SUGGEST_LOG_OP" for remove entries

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #49526 from dang/wip-58234-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:52:45 +0000 (11:52 -0800)]
Merge pull request #49526 from dang/wip-58234-pacific

pacific: rgw: Fix truncated ListBuckets response.

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #46106 from BenoitKnecht/wip-55500-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:52:15 +0000 (11:52 -0800)]
Merge pull request #46106 from BenoitKnecht/wip-55500-pacific

pacific: rgw: Avoid segfault when OPA authz is enabled

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #44471 from pritha-srivastava/wip-53648-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:51:13 +0000 (11:51 -0800)]
Merge pull request #44471 from pritha-srivastava/wip-53648-pacific

pacific: rgw/sts: fix read_obj_policy permission evaluation

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #44464 from pritha-srivastava/wip-52778-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:50:34 +0000 (11:50 -0800)]
Merge pull request #44464 from pritha-srivastava/wip-52778-pacific

pacific: rgw/sts: code to fetch certs using .well-known/openid-configuration URL

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #44463 from pritha-srivastava/wip-52785-pacific
Yuri Weinstein [Tue, 28 Nov 2023 19:49:20 +0000 (11:49 -0800)]
Merge pull request #44463 from pritha-srivastava/wip-52785-pacific

pacific: rgw/sts: fixes getsessiontoken authenticated with LDAP,

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
19 months agoMerge pull request #53693 from pdvian/wip-62996-pacific
Aishwarya Mathuria [Tue, 28 Nov 2023 16:31:24 +0000 (22:01 +0530)]
Merge pull request #53693 from pdvian/wip-62996-pacific

pacific: osd/OpRequest: Add detail description for delayed op in osd log file

19 months agoMerge pull request #54434 from ifed01/wip-ifed-no-alloc-lba-align-pac
Yuri Weinstein [Tue, 28 Nov 2023 16:02:26 +0000 (08:02 -0800)]
Merge pull request #54434 from ifed01/wip-ifed-no-alloc-lba-align-pac

pacific: os/bluestore: get rid off resulting lba alignment in allocators

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
19 months agoMerge pull request #53464 from k0ste/wip-56649-pacific
Yuri Weinstein [Tue, 28 Nov 2023 16:01:18 +0000 (08:01 -0800)]
Merge pull request #53464 from k0ste/wip-56649-pacific

pacific: pybind/mgr/autoscaler: Donot show NEW PG_NUM value if autoscaler is not on

Reviewed-by: Neha Ojha <nojha@redhat.com>
19 months agoMerge pull request #52203 from trociny/wip-61822-pacific
Yuri Weinstein [Tue, 28 Nov 2023 16:00:49 +0000 (08:00 -0800)]
Merge pull request #52203 from trociny/wip-61822-pacific

pacific: tools/osdmaptool: fix possible segfaults when there are down osds

Reviewed-by: Neha Ojha <nojha@redhat.com>
19 months agoMerge pull request #51262 from k0ste/wip-52841-pacific
Yuri Weinstein [Tue, 28 Nov 2023 16:00:22 +0000 (08:00 -0800)]
Merge pull request #51262 from k0ste/wip-52841-pacific

pacific: osd: fix shard-threads cannot wakeup bug

Reviewed-by: Neha Ojha <nojha@redhat.com>
19 months agoMerge pull request #50552 from rzarzynski/wip-59085-pacific
Yuri Weinstein [Tue, 28 Nov 2023 15:59:51 +0000 (07:59 -0800)]
Merge pull request #50552 from rzarzynski/wip-59085-pacific

pacific: mon: add proxy to cache tier options

Reviewed-by: Neha Ojha <nojha@redhat.com>
19 months agoMerge pull request #50194 from trociny/wip-58805-pacific
Yuri Weinstein [Tue, 28 Nov 2023 15:58:46 +0000 (07:58 -0800)]
Merge pull request #50194 from trociny/wip-58805-pacific

pacific: mgr: don't dump global config holding gil

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
19 months agomgr/dashboard: fix constraints.txt 54652/head
Ernesto Puerta [Wed, 22 Feb 2023 10:31:47 +0000 (11:31 +0100)]
mgr/dashboard: fix constraints.txt

Replaces strict version matching with compatible version syntax.

Reference: https://peps.python.org/pep-0440/#version-specifiers

Fixes: https://tracker.ceph.com/issues/58827
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit d5f8927a5cd4bfd9870f02fea1c66b32191d1fba)

19 months agoMerge pull request #54192 from cfsnyder/wip-62621-pacific
Nizamudeen A [Fri, 24 Nov 2023 05:02:09 +0000 (10:32 +0530)]
Merge pull request #54192 from cfsnyder/wip-62621-pacific

pacific: mgr/dashboard: add 'omit_usage' query param to dashboard api 'get rbd' endpoint

Reviewed-by: Nizamudeen A <nia@redhat.com>
19 months agomds: ensure next replay is queued on req drop 54314/head
Patrick Donnelly [Fri, 15 Jul 2022 20:39:00 +0000 (16:39 -0400)]
mds: ensure next replay is queued on req drop

Not all client replay requests are queued at once since [1]. We require
the next request by queued when completed (unsafely) or during cleanup.
Not all code paths seem to handle this [2] so move it to a generic
location, MDCache::request_cleanup. Even so, this doesn't handle all
errors (so we must still be careful) as sometimes we must queue the next
replay request before an MDRequest is constructed [3] during some error
conditions.

Additionally, preserve the behavior of Server::journal_and_reply
queueing the next replay op. Otherwise, must wait for the request to be
durable before moving onto the next one, unnecessarily.

For reproducing, two specific cases are highlighted (thanks to @Mer1997 on
Github for locating these):

- The request is killed by a session close / eviction while a replayed request
  is queued and waiting for a journal flush (e.g. dirty inest locks).

- The request construction fails because the request is already in the
  active_requests. This could happen theoretically if a client resends the same
  request (same reqid) twice.

The first case is most probable but very difficult to reproduce for testing
purposes. The replayed op would need to wait on a journal flush (to be
restarted by C_MDS_RetryRequest).  Then, the request would need killed by a
session close.

[1] ed6a18d90fdd1dc869369fb92c2aad43bc5c9a34
[2] https://github.com/ceph/ceph/blob/a6f1a1c6c09d74f5918c715b05789f34f2ea0e90/src/mds/Server.cc#L2253-L2262
[3] https://github.com/ceph/ceph/blob/a6f1a1c6c09d74f5918c715b05789f34f2ea0e90/src/mds/Server.cc#L2380

Fixes: https://tracker.ceph.com/issues/56577
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 078ecaa42b98f9858d2e3a045aedb51153b39e34)

Conflicts:
src/mds/Mutation.h: lock dump changes not backported
src/mds/Server.cc: minor code change

19 months agoMerge pull request #54410 from mihalicyn/wip-63478-pacific
Yuri Weinstein [Wed, 22 Nov 2023 20:01:19 +0000 (12:01 -0800)]
Merge pull request #54410 from mihalicyn/wip-63478-pacific

pacific: MClientRequest: properly handle ceph_mds_request_head_legacy for ext_num_retry, ext_num_fwd, owner_uid, owner_gid

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #54318 from batrick/wip-63414-pacific
Yuri Weinstein [Wed, 22 Nov 2023 20:00:39 +0000 (12:00 -0800)]
Merge pull request #54318 from batrick/wip-63414-pacific

pacific: mon: fix mds metadata lost in one case.

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #54245 from vshankar/wip-63283-pacific
Yuri Weinstein [Wed, 22 Nov 2023 19:58:26 +0000 (11:58 -0800)]
Merge pull request #54245 from vshankar/wip-63283-pacific

pacific: client: always refresh mds feature bits on session open

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #53716 from leonid-s-usov/bp/msg-wait/pacific
Yuri Weinstein [Wed, 22 Nov 2023 19:57:10 +0000 (11:57 -0800)]
Merge pull request #53716 from leonid-s-usov/bp/msg-wait/pacific

pacific: msg/AsyncMessenger: re-evaluate the stop condition when woken up in 'wait()'

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
19 months agoMerge pull request #53634 from vshankar/wip-62584-pacific
Yuri Weinstein [Wed, 22 Nov 2023 19:56:06 +0000 (11:56 -0800)]
Merge pull request #53634 from vshankar/wip-62584-pacific

pacific: mds: blocklist clients with "bloated" session metadata

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #53552 from batrick/wip-62906-pacific
Yuri Weinstein [Wed, 22 Nov 2023 19:55:20 +0000 (11:55 -0800)]
Merge pull request #53552 from batrick/wip-62906-pacific

pacific: mds,qa: some balancer debug messages (<=5) not printed when debug_mds is >=5

Reviewed-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #54604 from lxbsz/wip-63586-p
Ilya Dryomov [Wed, 22 Nov 2023 11:22:52 +0000 (12:22 +0100)]
Merge pull request #54604 from lxbsz/wip-63586-p

pacific: osd: log the number of extents for sparse read

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
19 months agoosd: add more debug logs for sparse read 54604/head
Xiubo Li [Mon, 20 Nov 2023 14:04:12 +0000 (22:04 +0800)]
osd: add more debug logs for sparse read

This will be very important to get to know what exactly has happened
when client get a very large number of extents.

URL: https://tracker.ceph.com/issues/63586
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 1cf5ecb93fb73a5ed4a82935f635f98efccf7de6)

19 months agoosd: don't require RWEXCL lock for stat+write ops. 54593/head
Alice Zhao [Mon, 11 Apr 2022 12:35:57 +0000 (08:35 -0400)]
osd: don't require RWEXCL lock for stat+write ops.
In librbd, a stat op is inserted before write op for cloned image. OSD used to use RWEXCL and such requests are processed one by one.
With this fix, OSD will use RWWRITE rather than RWEXCL for such [stat,write] request to allow multiple [stat,write] on the same object and improve performance.

Signed-off-by: Alice Zhao <brucen1030@163.com>
(cherry picked from commit 9be266b0a0304aaaaa0ca12f28fcd3e9cce1f9d7)

Conflicts:
src/osd/PrimaryLogPG.cc [ commit dfca9f8c0d8b ("osd: move
  PrimaryLogPG::get_rw_locks() from header to .cc.") not in
  pacific ]

19 months agoMerge pull request #54542 from aaSharma14/wip-63570-pacific
Aashish Sharma [Tue, 21 Nov 2023 11:17:36 +0000 (16:47 +0530)]
Merge pull request #54542 from aaSharma14/wip-63570-pacific

pacific: mgr/dashboard: Consider null values as zero in grafana panels

Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agomgr/dashboard: Consider null values as zero in grafana panels 54542/head
Aashish Sharma [Wed, 4 Oct 2023 06:54:13 +0000 (12:24 +0530)]
mgr/dashboard: Consider null values as zero in grafana panels

After upgrading from RHCS4 to RHCS5..some of the grafana charts broke.
This is because in RHCS5 we do not generate the metrics if its value is
zero as a result the null value from that metric breaks the grafana
charts or graphs. This PR is to fix the above mentioned issue.

Fixes: https://tracker.ceph.com/issues/63088
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 6f3f58cb8e4ce100cc7186858465b4d11d5c2c49)

20 months agoMerge pull request #54010 from k0ste/wip-53152-pacific
Yuri Weinstein [Wed, 15 Nov 2023 22:24:38 +0000 (14:24 -0800)]
Merge pull request #54010 from k0ste/wip-53152-pacific

pacific: radosgw-admin: allow 'bi purge' to delete index if entrypoint doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #54160 from cbodley/wip-61351-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:47:05 +0000 (13:47 -0800)]
Merge pull request #54160 from cbodley/wip-61351-pacific

pacific: rgw/s3: ListObjectsV2 returns correct object owners

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53764 from cbodley/wip-63043-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:46:11 +0000 (13:46 -0800)]
Merge pull request #53764 from cbodley/wip-63043-pacific

pacific: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53443 from trociny/wip-61872-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:45:17 +0000 (13:45 -0800)]
Merge pull request #53443 from trociny/wip-61872-pacific

pacific: rgw: pick http_date in case of http_x_amz_date absence

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52996 from k0ste/wip-59026-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:44:30 +0000 (13:44 -0800)]
Merge pull request #52996 from k0ste/wip-59026-pacific

pacific: rgw: use unique_ptr for flat_map emplace in BucketTrimWatche

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52605 from cbodley/wip-62138-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:20:33 +0000 (13:20 -0800)]
Merge pull request #52605 from cbodley/wip-62138-pacific

pacific: rgw: s3 object lock avoids overflow in retention date

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #54016 from k0ste/wip-57199-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:06:05 +0000 (13:06 -0800)]
Merge pull request #54016 from k0ste/wip-57199-pacific

pacific: rgw: 'bucket check' deletes index of multipart meta when its pending_map is nonempty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #54014 from k0ste/wip-59361-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:05:32 +0000 (13:05 -0800)]
Merge pull request #54014 from k0ste/wip-59361-pacific

pacific: rgw: fix rgw cache invalidation after unregister_watch() error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53775 from cbodley/wip-63055-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:04:55 +0000 (13:04 -0800)]
Merge pull request #53775 from cbodley/wip-63055-pacific

pacific: rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53772 from cbodley/wip-63052-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:04:10 +0000 (13:04 -0800)]
Merge pull request #53772 from cbodley/wip-63052-pacific

pacific: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53769 from cbodley/wip-63049-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:03:19 +0000 (13:03 -0800)]
Merge pull request #53769 from cbodley/wip-63049-pacific

pacific: rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53759 from cbodley/wip-63058-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:02:30 +0000 (13:02 -0800)]
Merge pull request #53759 from cbodley/wip-63058-pacific

pacific: rgw: fix unwatch crash at radosgw startup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53593 from trociny/wip-58478-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:01:45 +0000 (13:01 -0800)]
Merge pull request #53593 from trociny/wip-58478-pacific

pacific: rgw: fix FP error when calculating enteries per bi shard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53474 from k0ste/wip-55701-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:01:16 +0000 (13:01 -0800)]
Merge pull request #53474 from k0ste/wip-55701-pacific

pacific: radosgw-admin: don't crash on --placement-id without --storage-class

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53472 from k0ste/wip-57635-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:00:49 +0000 (13:00 -0800)]
Merge pull request #53472 from k0ste/wip-57635-pacific

pacific: rgw: Drain async_processor request queue during shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53439 from k0ste/wip-62823-pacific
Yuri Weinstein [Wed, 15 Nov 2023 21:00:19 +0000 (13:00 -0800)]
Merge pull request #53439 from k0ste/wip-62823-pacific

pacific: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53410 from trociny/wip-62308-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:59:46 +0000 (12:59 -0800)]
Merge pull request #53410 from trociny/wip-62308-pacific

pacific: rgw/sync-policy: Correct "sync status" & "sync group" commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53400 from trociny/wip-62751-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:59:14 +0000 (12:59 -0800)]
Merge pull request #53400 from trociny/wip-62751-pacific

pacific: rgw: fix 2 null versionID after convert_plain_entry_to_versioned

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #53376 from jzhu116-bloomberg/wip-59692-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:51:27 +0000 (12:51 -0800)]
Merge pull request #53376 from jzhu116-bloomberg/wip-59692-pacific

pacific: rgw/notification: remove non x-amz-meta-* attributes from bucket notifications

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
20 months agoMerge pull request #53356 from k0ste/wip-53658-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:50:05 +0000 (12:50 -0800)]
Merge pull request #53356 from k0ste/wip-53658-pacific

pacific: rgw: fix UploadPartCopy error code when src object not exist and src bucket not exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52936 from k0ste/wip-58902-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:49:17 +0000 (12:49 -0800)]
Merge pull request #52936 from k0ste/wip-58902-pacific

pacific: rgw: Fix Browser POST content-length-range min value

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52797 from cbodley/wip-62300-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:48:47 +0000 (12:48 -0800)]
Merge pull request #52797 from cbodley/wip-62300-pacific

pacific: rgw: retry metadata cache notifications with INVALIDATE_OBJ

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52729 from theanalyst/wip-58817
Yuri Weinstein [Wed, 15 Nov 2023 20:47:48 +0000 (12:47 -0800)]
Merge pull request #52729 from theanalyst/wip-58817

pacific: rgw: swift : check for valid key in POST forms

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #52113 from cbodley/wip-61728-pacific
Yuri Weinstein [Wed, 15 Nov 2023 20:45:37 +0000 (12:45 -0800)]
Merge pull request #52113 from cbodley/wip-61728-pacific

pacific: rgw/beast: add max_header_size option with 16k default, up from 4k

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #54167 from cbodley/wip-58238-pacific
Yuri Weinstein [Wed, 15 Nov 2023 16:28:52 +0000 (08:28 -0800)]
Merge pull request #54167 from cbodley/wip-58238-pacific

pacific: rgw: beast frontend checks for local_endpoint() errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #54120 from dparmar18/wip-63269-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:55:34 +0000 (07:55 -0800)]
Merge pull request #54120 from dparmar18/wip-63269-pacific

pacific: mds: report clients laggy due laggy OSDs only after checking any OSD is laggy

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53916 from kotreshhr/wip-63164-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:55:00 +0000 (07:55 -0800)]
Merge pull request #53916 from kotreshhr/wip-63164-pacific

pacific: pybind/mgr/volumes: log mutex locks to help debug deadlocks

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53576 from mchangir/wip-57157-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:54:26 +0000 (07:54 -0800)]
Merge pull request #53576 from mchangir/wip-57157-pacific

pacific: doc/cephfs: note regarding start time time zone

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
20 months agoMerge pull request #53556 from batrick/wip-62731-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:53:11 +0000 (07:53 -0800)]
Merge pull request #53556 from batrick/wip-62731-pacific

pacific: mds: add event for batching getattr/lookup

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53555 from batrick/wip-62897-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:52:40 +0000 (07:52 -0800)]
Merge pull request #53555 from batrick/wip-62897-pacific

pacific: qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53550 from batrick/wip-62902-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:51:56 +0000 (07:51 -0800)]
Merge pull request #53550 from batrick/wip-62902-pacific

pacific: mds: log message when exiting due to asok command

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
20 months agoMerge pull request #53495 from lxbsz/wip-62859
Yuri Weinstein [Wed, 15 Nov 2023 15:51:08 +0000 (07:51 -0800)]
Merge pull request #53495 from lxbsz/wip-62859

pacific: mds: fix deadlock between unlinking and linkmerge

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53486 from batrick/wip-62854-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:50:36 +0000 (07:50 -0800)]
Merge pull request #53486 from batrick/wip-62854-pacific

pacific: qa: ignore expected cluster warning from damage tests

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53453 from joscollin/wip-62834-pacific
Yuri Weinstein [Wed, 15 Nov 2023 15:49:44 +0000 (07:49 -0800)]
Merge pull request #53453 from joscollin/wip-62834-pacific

pacific: cephfs-top: include the missing fields in --dump output

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoos/bluestore: proper init sequent in BlueFS::mount() 54434/head
Igor Fedotov [Tue, 14 Nov 2023 13:49:28 +0000 (16:49 +0300)]
os/bluestore: proper init sequent in BlueFS::mount()

Requires in pacific backport only due to the lack of
https://github.com/ceph/ceph/pull/41557

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
20 months agoMerge PR #52852 into pacific
Patrick Donnelly [Mon, 13 Nov 2023 14:46:16 +0000 (09:46 -0500)]
Merge PR #52852 into pacific

* refs/pull/52852/head:
mds: remove calculating caps after adding revokes back
test/libcephfs: add test case for revoking caps
client: issue a cap release immediately if no cap exists
mds: add the revoking caps back to _revokes list
mds: move confirm_receipt() to Capability.cc

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
20 months agodoc/rados/configuration/bluestore-config-ref: Fix lowcase typo 54474/head
Adam Kupczyk [Fri, 3 Nov 2023 15:50:19 +0000 (16:50 +0100)]
doc/rados/configuration/bluestore-config-ref: Fix lowcase typo

Letters "L P" got lowcased to "l p".
Reverted that.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
(cherry picked from commit 59a7f01b5c3cc48aa1e15ee44204d3552ffd593a)

20 months agoclient: queue a delay cap flushing if there are ditry caps/snapcaps 54472/head
Xiubo Li [Thu, 21 Sep 2023 07:02:38 +0000 (15:02 +0800)]
client: queue a delay cap flushing if there are ditry caps/snapcaps

We should queue a cap release anyway if there are dirty caps/snapcaps
for the inodes when trimming caps.

Fixes: https://tracker.ceph.com/issues/62979
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit d14752ff1f56da93ad0d9c94fcf101945fdacadd)

20 months agoqa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted 53634/head
Venky Shankar [Mon, 9 Oct 2023 04:29:45 +0000 (09:59 +0530)]
qa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5856a1e6b72dc77d3b4d9109385481b924e4fe36)

20 months agoqa: lower mds_session_metadata_threshold for tests
Venky Shankar [Sat, 7 Oct 2023 15:12:29 +0000 (20:42 +0530)]
qa: lower mds_session_metadata_threshold for tests

... and increase the number of files that are created so as to
hit the threshold with a high probability.

Fixes: http://tracker.ceph.com/issues/62873
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 92200d9d10d30ed35f640fd1120be65577a3ce64)

20 months agoPendingReleaseNotes: add a note for `mds_session_metadata_threshold` mds config
Venky Shankar [Thu, 17 Aug 2023 04:40:37 +0000 (10:10 +0530)]
PendingReleaseNotes: add a note for `mds_session_metadata_threshold` mds config

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit ac3ab1a203ad65c7db07bc2c02acb5620800fb25)