]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 months agorgw/auth: ignoring signatures for HTTP OPTIONS calls 60458/head
Tobias Urdin [Tue, 6 Feb 2024 07:50:55 +0000 (07:50 +0000)]
rgw/auth: ignoring signatures for HTTP OPTIONS calls

Before [1] we always sent all HTTP OPTIONS requests to
the S3AnonymousEngine and ignored any provided AWSv4
credentials sent in the request.

That PR changed so that if we got credentials in the
request we instead sent it through the authentication
code in order to solve HTTP OPTIONS requests on tenanted
users to start working (because we need to resolve the
tenant, also called bucket tenant in the code, and we can't
only rely on the bucket name since it will not be found).

We solved this by modifying the canonical HTTP method used
when calculating the AWSv4 signature by instead using the
access-control-request-method header which worked good.

This change did not take into account that when you generated
a presigned URL for a put_object request you can also pass in
extra parameters like a canned ACL [2] to the Params variable
in for example boto3's generated_presigned_url().

Doing that will cause the client to add the x-amz-acl header
to x-amz-signedheaders and also use that in their signature
calculation.

When doing a HTTP OPTIONS calls for CORS on that presigned URL
the browser will never send a x-amz-acl header with the correct
data since that is something that the actual PUT request should
include later, so that HTTP OPTIONS call should pass even though
the signature can never be calculated correctly server-side like
verified against AWS S3 in tracker [3].

This patch as a result skips the signature calculation when doing
EC2 auth using the LocalEngine but we still need to pass the request
there in order to lookup the user to support buckets in a tenant.

For the Keystone EC2 auth we're pretty out of luck in the sense that
Keystone's API itself requires us to send the AWSv4 signature in the
request with the access_key in order to obtain a token, and we cannot
leave the signature out, we also cannot spoof the signature from
rgw -> keystone since we don't have access to the secret_key if it's
not in our cache.

For that approach we simply pass on to get_access_token() that if it's
an HTTP OPTIONS and we find the access_key in the cache we pull that
and ignore verifying signature and pass it on for validation. This means
that the cache must be warm if using Keystone auth and adding extra
params to a presigned URL.

This partly makes some of the commits in [1] redundant for EC2
LocalEngine auth but we still need it for tenanted bucket support.

[1] https://github.com/ceph/ceph/pull/52673
[2] https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl
[3] https://tracker.ceph.com/issues/64308

Fixes: https://tracker.ceph.com/issues/64308
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
(cherry picked from commit fe15b52edb5d228d2ed56679c62cf48493ae2d54)

Conflicts: no optional_yield argument in quincy
src/rgw/rgw_auth_keystone.cc
src/rgw/rgw_auth_keystone.h

8 months agoMerge pull request #60437 from zdover23/wip-doc-2024-10-23-backport-60433-to-quincy
Anthony D'Atri [Wed, 23 Oct 2024 01:50:59 +0000 (21:50 -0400)]
Merge pull request #60437 from zdover23/wip-doc-2024-10-23-backport-60433-to-quincy

quincy: doc/cephfs: rearrange subvolume group information

8 months agodoc/cephfs: rearrange subvolume group information 60437/head
Indira Sawant [Tue, 22 Oct 2024 21:45:10 +0000 (16:45 -0500)]
doc/cephfs: rearrange subvolume group information

Rearranging the documentation to align it with CephFS encapsulation design.

Signed-off-by: Indira Sawant <indira.sawant@ibm.com>
(cherry picked from commit f6cf7ae5281f5740ac78bfa5e2a22913353b8167)

8 months agoMerge pull request #60314 from vshankar/wip-client-secfix
Yuri Weinstein [Tue, 22 Oct 2024 01:59:18 +0000 (18:59 -0700)]
Merge pull request #60314 from vshankar/wip-client-secfix

quincy: client: disallow unprivileged users to escalate root privileges

Reviewed-by: Venky Shankar vshankar@redhat.com
8 months agoMerge pull request #58843 from lxbsz/wip-67163
Yuri Weinstein [Mon, 21 Oct 2024 18:05:29 +0000 (11:05 -0700)]
Merge pull request #58843 from lxbsz/wip-67163

quincy: mds: try to choose a new batch head in request_clientup()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
8 months agoMerge pull request #58296 from batrick/wip-66624-quincy
Yuri Weinstein [Mon, 21 Oct 2024 18:03:43 +0000 (11:03 -0700)]
Merge pull request #58296 from batrick/wip-66624-quincy

quincy: mds: do remove the cap when seqs equal or larger than last issue

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #54549 from k0ste/wip-62817-quincy
Yuri Weinstein [Fri, 18 Oct 2024 15:07:30 +0000 (08:07 -0700)]
Merge pull request #54549 from k0ste/wip-62817-quincy

quincy: osd: ensure async recovery does not drop a pg below min_size

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #60124 from adamemerson/wip-40692-quincy
Yuri Weinstein [Thu, 17 Oct 2024 14:13:05 +0000 (07:13 -0700)]
Merge pull request #60124 from adamemerson/wip-40692-quincy

quincy: global: Call getnam_r with a 64KiB buffer on the heap

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
9 months agoMerge pull request #59498 from pponnuvel/wip-67606-quincy
Yuri Weinstein [Thu, 17 Oct 2024 14:12:00 +0000 (07:12 -0700)]
Merge pull request #59498 from pponnuvel/wip-67606-quincy

quincy: os/bluestore: allow use BtreeAllocator

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
9 months agoMerge pull request #57406 from k0ste/wip-58815-quincy
Yuri Weinstein [Thu, 17 Oct 2024 14:11:19 +0000 (07:11 -0700)]
Merge pull request #57406 from k0ste/wip-58815-quincy

quincy: rados: Set snappy as default value in ms_osd_compression_algorithm

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #57363 from batrick/wip-65871-quincy
Yuri Weinstein [Thu, 17 Oct 2024 14:10:18 +0000 (07:10 -0700)]
Merge pull request #57363 from batrick/wip-65871-quincy

quincy: common/StackStringStream: update pointer to newly allocated memory in overflow()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
9 months agoMerge pull request #54557 from k0ste/wip-57795-quincy
Yuri Weinstein [Thu, 17 Oct 2024 14:08:42 +0000 (07:08 -0700)]
Merge pull request #54557 from k0ste/wip-57795-quincy

quincy: common:  intrusive_lru destructor add

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #60337 from dmick/wip-quincy-container-ceph-node-proxy
Dan Mick [Wed, 16 Oct 2024 18:22:04 +0000 (11:22 -0700)]
Merge pull request #60337 from dmick/wip-quincy-container-ceph-node-proxy

quincy: ceph-node-proxy not present, not part of container

9 months agoMerge pull request #60334 from cbodley/wip-doc-rgw-multisite-secondary-commit-quincy
Casey Bodley [Wed, 16 Oct 2024 15:05:03 +0000 (11:05 -0400)]
Merge pull request #60334 from cbodley/wip-doc-rgw-multisite-secondary-commit-quincy

quincy: doc/radosgw/multisite: fix Configuring Secondary Zones -> Updating the Period

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
9 months agoMerge pull request #56702 from joscollin/wip-65320-quincy
Venky Shankar [Wed, 16 Oct 2024 13:30:39 +0000 (19:00 +0530)]
Merge pull request #56702 from joscollin/wip-65320-quincy

quincy: cephfs-mirror: various fixes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58333 from mchangir/wip-66671-quincy
Venky Shankar [Wed, 16 Oct 2024 13:30:24 +0000 (19:00 +0530)]
Merge pull request #58333 from mchangir/wip-66671-quincy

quincy: qa: cleanup snapshots before subvolume delete

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58724 from neesingh-rh/wip-67076-quincy
Venky Shankar [Wed, 16 Oct 2024 13:30:13 +0000 (19:00 +0530)]
Merge pull request #58724 from neesingh-rh/wip-67076-quincy

quincy: mds: fix session/client evict command.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58773 from lxbsz/wip-66668
Venky Shankar [Wed, 16 Oct 2024 13:29:57 +0000 (18:59 +0530)]
Merge pull request #58773 from lxbsz/wip-66668

quincy: mds: set the correct WRLOCK flag always in wrlock_force()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58799 from neesingh-rh/wip-66870-quincy
Venky Shankar [Wed, 16 Oct 2024 13:29:40 +0000 (18:59 +0530)]
Merge pull request #58799 from neesingh-rh/wip-66870-quincy

quincy: mds: prevent scrubbing for standby-replay MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58806 from lxbsz/wip-67150
Venky Shankar [Wed, 16 Oct 2024 13:29:28 +0000 (18:59 +0530)]
Merge pull request #58806 from lxbsz/wip-67150

quincy: cephfs: Fixed a bug in the readdir_cache_cb function that may have us…

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #58807 from lxbsz/wip-67119
Venky Shankar [Wed, 16 Oct 2024 12:59:55 +0000 (18:29 +0530)]
Merge pull request #58807 from lxbsz/wip-67119

quincy: qa: ignore container checkpoint/restore related selinux denials for c…

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #59037 from vshankar/wip-quincy-rm-centos8
Venky Shankar [Wed, 16 Oct 2024 12:59:17 +0000 (18:29 +0530)]
Merge pull request #59037 from vshankar/wip-quincy-rm-centos8

quincy: qa/cephfs: no reliance on centos

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge pull request #59067 from joscollin/wip-67381-quincy
Venky Shankar [Wed, 16 Oct 2024 12:30:52 +0000 (18:00 +0530)]
Merge pull request #59067 from joscollin/wip-67381-quincy

quincy: qa/suites: drop --show-reachable=yes from fs:valgrind tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoquincy: ceph-node-proxy not present, not part of container 60337/head
Dan Mick [Tue, 15 Oct 2024 19:55:20 +0000 (12:55 -0700)]
quincy: ceph-node-proxy not present, not part of container

Signed-off-by: Dan Mick <dmick@redhat.com>
9 months agodoc/radosgw/multisite: fix Configuring Secondary Zones -> Updating the Period 60334/head
Casey Bodley [Fri, 4 Oct 2024 19:50:05 +0000 (15:50 -0400)]
doc/radosgw/multisite: fix Configuring Secondary Zones -> Updating the Period

this was copy/pasted from Configuring a Master Zone -> Update the Period
but still referred to the 'master zone'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cfe254758b1ca9647c4dcfb13b6a3310558b88d2)

9 months agoMerge pull request #59403 from Matan-B/wip-67665-quincy
SrinivasaBharathKanta [Tue, 15 Oct 2024 14:12:26 +0000 (19:42 +0530)]
Merge pull request #59403 from Matan-B/wip-67665-quincy

quincy: mon/OSDMonitor: Add force-remove-snap mon command

9 months agoMerge pull request #59308 from batrick/wip-67476-quincy
SrinivasaBharathKanta [Tue, 15 Oct 2024 14:12:08 +0000 (19:42 +0530)]
Merge pull request #59308 from batrick/wip-67476-quincy

quincy: mon/MonClient: handle ms_handle_fast_authentication return

9 months agoMerge pull request #59278 from k0ste/wip-67240-quincy
SrinivasaBharathKanta [Tue, 15 Oct 2024 14:11:49 +0000 (19:41 +0530)]
Merge pull request #59278 from k0ste/wip-67240-quincy

quincy: mon: fix inconsistencies in class param

9 months agoMerge pull request #60201 from zdover23/wip-doc-2024-10-09-backport-60163-to-quincy
Zac Dover [Tue, 15 Oct 2024 12:20:05 +0000 (22:20 +1000)]
Merge pull request #60201 from zdover23/wip-doc-2024-10-09-backport-60163-to-quincy

quincy: doc/man: supplant "wsync" with "nowsync" as the default

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
9 months agoMerge pull request #55216 from ifed01/wip-ifed-refactor-spillover-tests-qui
Igor Fedotov [Tue, 15 Oct 2024 09:38:32 +0000 (12:38 +0300)]
Merge pull request #55216 from ifed01/wip-ifed-refactor-spillover-tests-qui

quincy: test/store_test: refactor spillover tests

Reviewed-by: Pere Diaz Bou <pere-altea@hotmail.com>
9 months agoclient: disallow unprivileged users to escalate root privileges 60314/head
Xiubo Li [Wed, 3 Apr 2024 11:02:08 +0000 (19:02 +0800)]
client: disallow unprivileged users to escalate root privileges

An unprivileged user can `chmod 777` a directory owned by root
and gain access. Fix this bug and also add a test case for the
same.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
9 months agotest/store_test: refactor spillover tests 55216/head
Igor Fedotov [Thu, 19 Oct 2023 11:59:16 +0000 (14:59 +0300)]
test/store_test: refactor spillover tests

Fixes: https://tracker.ceph.com/issues/62401
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 5019495659359bbc19c8ab3988e3d9e12c5f58c1)

 Conflicts:
src/test/objectstore/store_test.cc
 * trivial

9 months agoos/bluestore: provide some reference for BlueFSVolumeSelector's interface.
Igor Fedotov [Wed, 8 Nov 2023 21:22:28 +0000 (00:22 +0300)]
os/bluestore: provide some reference for BlueFSVolumeSelector's interface.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 243d21ef1c18d561b35acf309e8e1c13b0781658)

9 months agoos/bluestore: rework vselector calls
Igor Fedotov [Thu, 19 Oct 2023 11:57:23 +0000 (14:57 +0300)]
os/bluestore: rework vselector calls

We can provide fnode delta to vseector now. Which is a bit more
effective.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 3e04ac48779f62394ebd9587298823bf6b20a570)

 Conflicts:
src/os/bluestore/BlueFS.cc
 * trival

9 months agoMerge pull request #60230 from dmick/wip-68470-quincy
Laura Flores [Mon, 14 Oct 2024 23:00:09 +0000 (18:00 -0500)]
Merge pull request #60230 from dmick/wip-68470-quincy

quincy: Add Containerfile and build.sh to build it.

9 months agoMerge pull request #59442 from amathuria/wip-65946-quincy
Yuri Weinstein [Mon, 14 Oct 2024 17:49:20 +0000 (10:49 -0700)]
Merge pull request #59442 from amathuria/wip-65946-quincy

quincy: msg/async: Encode message once features are set

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #54356 from leonid-s-usov/bp/mds-raise/quincy
Yuri Weinstein [Mon, 14 Oct 2024 17:47:13 +0000 (10:47 -0700)]
Merge pull request #54356 from leonid-s-usov/bp/mds-raise/quincy

quincy: common/admin_socket: add a command to raise a signal

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #48310 from k0ste/wip-56637-quincy
Yuri Weinstein [Mon, 14 Oct 2024 17:46:35 +0000 (10:46 -0700)]
Merge pull request #48310 from k0ste/wip-56637-quincy

quincy: log: Make log_max_recent have an effect again.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agocontainer/build.sh: fix arm architecture tagging 60230/head
Dan Mick [Thu, 10 Oct 2024 23:38:58 +0000 (16:38 -0700)]
container/build.sh: fix arm architecture tagging

The wrong string was used for comparison, and for tagging,
so the arm64 branch and sha1 images overwrote and destroyed
the amd64 images.

Signed-off-by: Dan Mick <dmick@redhat.com>
(cherry picked from commit 7e03ee798f4ed3aa4a0bb1a9e9d62df52e54406d)

9 months agoMerge pull request #55775 from ifed01/wip-ifed-no-death-tests-qui
Yuri Weinstein [Mon, 14 Oct 2024 14:57:38 +0000 (07:57 -0700)]
Merge pull request #55775 from ifed01/wip-ifed-no-death-tests-qui

quincy: test/store_test: get rid off assert_death.

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
9 months agoMerge pull request #60299 from zdover23/wip-doc-2024-10-14-backport-60242-to-quincy
Anthony D'Atri [Mon, 14 Oct 2024 14:49:36 +0000 (10:49 -0400)]
Merge pull request #60299 from zdover23/wip-doc-2024-10-14-backport-60242-to-quincy

quincy: doc: SubmittingPatches-backports - remove backports team

9 months agoMerge pull request #60274 from zdover23/wip-doc-2024-10-12-backport-60243-to-quincy
Zac Dover [Mon, 14 Oct 2024 13:57:58 +0000 (23:57 +1000)]
Merge pull request #60274 from zdover23/wip-doc-2024-10-12-backport-60243-to-quincy

quincy: doc/dev: remove "Stable Releases and Backports"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
9 months agodoc: SubmittingPatches-backports - remove backports team 60299/head
Zac Dover [Thu, 10 Oct 2024 12:11:22 +0000 (22:11 +1000)]
doc: SubmittingPatches-backports - remove backports team

Remove all references to the "Stable Releases and Backports Team", which
as of October 2024 does not exist.

Fixes: https://tracker.ceph.com/issues/68471
Co-authored-by: Laura Flores <lflores@ibm.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0317b5f87ac22399f6242d72f0bb9924794687de)

9 months agodoc/dev: remove "Stable Releases and Backports" 60274/head
Zac Dover [Thu, 10 Oct 2024 12:17:12 +0000 (22:17 +1000)]
doc/dev: remove "Stable Releases and Backports"

Remove mention of "Stable Releases and Backports" from
doc/dev/developer_guide/essentials.rst.

Co-authored-by: Laura Flores <lflores@ibm.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 95916c91906604c516e78b550010e9de511fc1e9)

9 months agoMerge pull request #60271 from zdover23/wip-doc-2024-10-12-backport-59507-to-quincy
Anthony D'Atri [Fri, 11 Oct 2024 20:06:02 +0000 (16:06 -0400)]
Merge pull request #60271 from zdover23/wip-doc-2024-10-12-backport-59507-to-quincy

quincy: doc/rbd: add namespace information for mirror commands

9 months agodoc/rbd: add namespace information for mirror commands 60271/head
N Balachandran [Thu, 29 Aug 2024 09:04:44 +0000 (14:34 +0530)]
doc/rbd: add namespace information for mirror commands

Updates the rbd man page to clarify  that some mirror commands
can apply to pools and namespaces.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 5e64748927d029ceffc27edbc6cbf7c774a7b291)

9 months agoMerge pull request #60235 from zdover23/wip-doc-2024-10-10-backport-60204-to-quincy
Anthony D'Atri [Thu, 10 Oct 2024 12:45:16 +0000 (08:45 -0400)]
Merge pull request #60235 from zdover23/wip-doc-2024-10-10-backport-60204-to-quincy

quincy: doc: fix email

9 months agodoc: fix email 60235/head
Ernesto Puerta [Wed, 9 Oct 2024 06:12:42 +0000 (08:12 +0200)]
doc: fix email

Signed-off-by: Ernesto Puerta <37327689+epuertat@users.noreply.github.com>
(cherry picked from commit 512eea1af52126e9b082744ee7b870c12c23d55d)

9 months agoAdd Containerfile and build.sh to build it.
Dan Mick [Wed, 26 Jun 2024 02:07:41 +0000 (19:07 -0700)]
Add Containerfile and build.sh to build it.

The intent is to replace ceph-container.git, at first for ci containers
only, and eventually production containers as well.

There is code present for production containers, including
a separate "make-manifest-list.py" to scan for and glue the two
arch-specific containers into a 'manifest-list' 'fat' container,
but that code is not yet fully tested.

This code will not be used until a corresponding change to the
Jenkins jobs in ceph-build.git is pushed.

Note that this tooling does not authenticate to the container repo;
it is assumed that will be done elsewhere.  Authentication is
verified by pushing a minimal image to the requested repo.

Signed-off-by: Dan Mick <dmick@redhat.com>
(cherry picked from commit 5c40a5c1737cf0628c899c4e9f788017b8b4fca8)

9 months agorados: Set snappy as default value in ms_osd_compression_algorithm instead of list... 57406/head
shreyanshjain7174 [Tue, 24 Jan 2023 14:11:58 +0000 (08:11 -0600)]
rados: Set snappy as default value in ms_osd_compression_algorithm instead of list of algorithms.

The OSDs choose the preferred algorithm based on mutual understanding between OSDs. The proposed solution to this problem was to set "snappy" as default algorithm and provide flexibility to user to change the algorithm of their choice.

Fixes: https://tracker.ceph.com/issues/58410
Signed-off-by: shreyanshjain7174 <ssanchet@redhat.com>
(cherry picked from commit 158986a3b73a42db8e31f3366912e906c743f145)

9 months agolog: Make log_max_recent have an effect again. 48310/head
Joshua Baergen [Thu, 16 Jun 2022 16:14:12 +0000 (10:14 -0600)]
log: Make log_max_recent have an effect again.

The log improvements in a747aeac13daf3dba43343120659e802cb569f6b
unfortunately left log_max_recent broken because m_max_recent wasn't
used anymore.

Eliminate m_max_recent and set the capacity of the m_recent ring buffer
when log_max_recent changes. In order to call set_capacity(),
ConcreteEntry needed its move constructor set noexcept.

I haven't followed the boost code all the way down but I suspect that
setting the ring buffer capacity to anything less than 1 entry will
probably cause problems, so restrict log_max_recent to >=1.

Also fix a wrong variable used for printing the max new entries during
"log dump".

Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
(cherry picked from commit 3d59ba1671e3e76326fb706a76b8d9638d782924)

9 months agoMerge pull request #55598 from NitzanMordhai/wip-63976-quincy
Yuri Weinstein [Tue, 8 Oct 2024 18:05:54 +0000 (11:05 -0700)]
Merge pull request #55598 from NitzanMordhai/wip-63976-quincy

quincy: Tools/rados: Improve Error Messaging for Object Name Resolution

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #55597 from NitzanMordhai/wip-63982-quincy
Yuri Weinstein [Tue, 8 Oct 2024 18:05:04 +0000 (11:05 -0700)]
Merge pull request #55597 from NitzanMordhai/wip-63982-quincy

quincy: mon/Monitor: during shutdown don't accept new authentication and crea…

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
9 months agoMerge pull request #54555 from k0ste/wip-57473-quincy
Yuri Weinstein [Tue, 8 Oct 2024 18:02:59 +0000 (11:02 -0700)]
Merge pull request #54555 from k0ste/wip-57473-quincy

quincy: mgr: fix a race condition in DaemonServer::handle_report()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #54012 from k0ste/wip-63036-quincy
Yuri Weinstein [Tue, 8 Oct 2024 18:02:17 +0000 (11:02 -0700)]
Merge pull request #54012 from k0ste/wip-63036-quincy

quincy: mgr: add throttle policy for DaemonServer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #50193 from trociny/wip-58806-quincy
Yuri Weinstein [Tue, 8 Oct 2024 18:01:13 +0000 (11:01 -0700)]
Merge pull request #50193 from trociny/wip-58806-quincy

quincy: mgr: don't dump global config holding gil

Reviewed-by: Tim Serong <tserong@suse.com>
9 months agodoc/man: supplant "wsync" with "nowsync" as the default 60201/head
Zac Dover [Mon, 7 Oct 2024 13:21:02 +0000 (23:21 +1000)]
doc/man: supplant "wsync" with "nowsync" as the default

Change the default from "wsync" to "nowsync". "nowsync" has been the
default since kernel version 5.16.

This information was reported on
https://pad.ceph.com/p/Report_Documentation_Bugs by Benjamin Mare on 07
Oct 2024.

The commit relevant to this change can be seen here, in Linus's main
branch:
https://github.com/torvalds/linux/commit/f7a67b463fb83a4b9b11ceaa8ec4950b8fb7f902

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9f0af5d75dc64ef0520fd78f29815d2a824a859b)

9 months agoMerge pull request #59523 from badone/wip-67822-quincy
Yuri Weinstein [Tue, 8 Oct 2024 14:59:10 +0000 (07:59 -0700)]
Merge pull request #59523 from badone/wip-67822-quincy

quincy: test/lazy-omap-stats: Convert to boost::regex

Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #60064 from sajibreadd/wip-68325-quincy
Yuri Weinstein [Mon, 7 Oct 2024 23:41:04 +0000 (16:41 -0700)]
Merge pull request #60064 from sajibreadd/wip-68325-quincy

quincy: common/Formatter: dump inf/nan as null

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
9 months agoMerge pull request #59532 from ifed01/wip-ifed-fix-spillover-qui
Yuri Weinstein [Mon, 7 Oct 2024 23:40:17 +0000 (16:40 -0700)]
Merge pull request #59532 from ifed01/wip-ifed-fix-spillover-qui

quincy: test/store_test: enforce sync compactions for spillover tests

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
9 months agoMerge pull request #59468 from sajibreadd/wip-67039-quincy
Yuri Weinstein [Mon, 7 Oct 2024 23:39:00 +0000 (16:39 -0700)]
Merge pull request #59468 from sajibreadd/wip-67039-quincy

quincy: os/bluestore: Warning added for slow operations and stalled read

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
9 months agoMerge pull request #58674 from ifed01/wip-ifed-many-many-extents-read-qui
Yuri Weinstein [Mon, 7 Oct 2024 23:38:00 +0000 (16:38 -0700)]
Merge pull request #58674 from ifed01/wip-ifed-many-many-extents-read-qui

quincy: blk/aio: fix long batch (64+K entries) submission

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
9 months agoMerge pull request #56138 from ifed01/wip-ifed-fix-global-repair-stats-qui
Yuri Weinstein [Mon, 7 Oct 2024 23:36:41 +0000 (16:36 -0700)]
Merge pull request #56138 from ifed01/wip-ifed-fix-global-repair-stats-qui

quincy: os/store_test: Retune tests to current code

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
9 months agoMerge pull request #55133 from ifed01/wip-ifed-refresh-perf-counters-qui
Yuri Weinstein [Mon, 7 Oct 2024 23:35:39 +0000 (16:35 -0700)]
Merge pull request #55133 from ifed01/wip-ifed-refresh-perf-counters-qui

quincy: os: introduce ObjectStore::refresh_perf_counters() method

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
9 months agoMerge pull request #59236 from NitzanMordhai/wip-67571-quincy
Radoslaw Zarzynski [Mon, 7 Oct 2024 19:58:33 +0000 (21:58 +0200)]
Merge pull request #59236 from NitzanMordhai/wip-67571-quincy

quincy: cls/cas/cls_cas_internal: Initialize 'hash' value before decoding

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #60136 from zdover23/wip-doc-2024-10-05-backport-60116-to-quincy
Anthony D'Atri [Sat, 5 Oct 2024 15:42:35 +0000 (11:42 -0400)]
Merge pull request #60136 from zdover23/wip-doc-2024-10-05-backport-60116-to-quincy

quincy: doc/governance: add Zac Dover's updated email

9 months agodoc/governance: add Zac Dover's updated email 60136/head
Zac Dover [Fri, 4 Oct 2024 13:25:30 +0000 (23:25 +1000)]
doc/governance: add Zac Dover's updated email

Update Zac Dover's email address.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 322c1a792062a72e958a9cfbd6a24a3e7333908d)

9 months agoglobal: Call getnam_r with a 64KiB buffer on the heap 60124/head
Adam Emerson [Wed, 2 Oct 2024 18:42:52 +0000 (14:42 -0400)]
global: Call getnam_r with a 64KiB buffer on the heap

Fixes: https://tracker.ceph.com/issues/40692
Signed-off-by: Adam Emerson <aemerson@redhat.com>
(cherry picked from commit 9eeca730fa0d99f39d81f6d16cb5faff9d033516)

Fixes: https://tracker.ceph.com/issues/68390
Conflicts:
src/global/global_init.cc
 - Replace `make_unique_for_overwrite` which is C++20 with `make_unique`

Signed-off-by: Adam Emerson <aemerson@redhat.com>
9 months agoMerge pull request #60087 from idryomov/wip-68213-quincy
Ilya Dryomov [Thu, 3 Oct 2024 14:07:15 +0000 (16:07 +0200)]
Merge pull request #60087 from idryomov/wip-68213-quincy

quincy: qa/suites/rbd/iscsi: enable all supported container hosts

Reviewed-by: N Balachandran <nibalach@redhat.com>
9 months agoqa/suites/rbd/iscsi: enable all supported container hosts 60087/head
Ilya Dryomov [Tue, 24 Sep 2024 12:06:42 +0000 (14:06 +0200)]
qa/suites/rbd/iscsi: enable all supported container hosts

On Ubuntu open-iscsi and multipath-tools need to be installed instead
of iscsi-initiator-utils and device-mapper-multipath.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 29e122f8754aa56a2b1cdc56ac4f0591828e5269)

Conflicts:
- .qa/distros/supported-container-hosts doesn't exist in quincy,
  replaced with .qa/distros/container-hosts to fix the symlink

9 months agoqa: avoid a non-standard shell construct in rbd/iscsi_client.t
Ilya Dryomov [Thu, 26 Sep 2024 15:36:37 +0000 (17:36 +0200)]
qa: avoid a non-standard shell construct in rbd/iscsi_client.t

dash which is used as /bin/sh on Ubuntu interprets "2&> /dev/null" as
an instruction to launch iscsiadm in the background.  While that is
mostly compensated by the following sleep, stderr isn't redirected to
/dev/null either -- the output gets polluted and the test fails.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d3f385aaed0b7631f6d69bf1ed88f9570b1d8f50)

9 months agoqa: don't hardcode podman in rbd/gwcli_{create,delete}.t
Ilya Dryomov [Wed, 25 Sep 2024 09:34:34 +0000 (11:34 +0200)]
qa: don't hardcode podman in rbd/gwcli_{create,delete}.t

If podman isn't available (e.g. on Ubuntu), cephadm falls back to
docker.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit b66b98e76d424b2881084d97d5da0e3c9ce062be)

9 months agoqa/tasks/ceph_iscsi_client: don't use mpathconf
Ilya Dryomov [Wed, 25 Sep 2024 07:53:35 +0000 (09:53 +0200)]
qa/tasks/ceph_iscsi_client: don't use mpathconf

... since it's not available on Ubuntu.  In this case mpathconf just
sets a couple of default values and defines an empty blacklist section,
so it's easy enough to replicate.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 9022ef19a58155620b3626080ea58b3c60b6fa36)

9 months agoMerge pull request #60086 from zdover23/wip-doc-2024-10-02-backport-59164-to-quincy
Anthony D'Atri [Tue, 1 Oct 2024 21:33:30 +0000 (17:33 -0400)]
Merge pull request #60086 from zdover23/wip-doc-2024-10-02-backport-59164-to-quincy

quincy: doc:update e-mail addresses governance

9 months agodoc:update e-mail addresses governance 60086/head
Tobias Fischer [Mon, 12 Aug 2024 09:30:48 +0000 (11:30 +0200)]
doc:update e-mail addresses governance

with the consent of my colleagues
- Dan van der Ster
- João Eduardo Luis
- Mark Nelson

changed their e-mail addresses to the current one

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
Signed-off-by: Tobias Fischer <138117964+toficly@users.noreply.github.com>
(cherry picked from commit 180378b5adee303fe57ecd4951b0f0ff46afa5e1)

9 months agoMerge pull request #58512 from k0ste/wip-66889-quincy
Yuri Weinstein [Tue, 1 Oct 2024 21:02:02 +0000 (14:02 -0700)]
Merge pull request #58512 from k0ste/wip-66889-quincy

quincy: mgr/Mgr.cc: clear daemon health metrics instead of removing down/out osd from daemon state

Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #58845 from idryomov/wip-58120-quincy
Yuri Weinstein [Tue, 1 Oct 2024 19:56:54 +0000 (12:56 -0700)]
Merge pull request #58845 from idryomov/wip-58120-quincy

quincy: osd: avoid watcher remains after "rados watch" is interrupted

Reviewed-by: Laura Flores <lflores@redhat.com>
9 months agoMerge pull request #60074 from kshtsk/wip-68349-quincy
kyr [Tue, 1 Oct 2024 15:53:01 +0000 (17:53 +0200)]
Merge pull request #60074 from kshtsk/wip-68349-quincy

quincy: ceph.spec.in: we need jsonnet for all distroes for make check

9 months agoMerge pull request #58921 from mohit84/wip-67236-quincy
Yuri Weinstein [Tue, 1 Oct 2024 15:14:57 +0000 (08:14 -0700)]
Merge pull request #58921 from mohit84/wip-67236-quincy

quincy: test: Create ParallelPGMapper object before start threadpool

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
9 months agoMerge pull request #59370 from NitzanMordhai/wip-67643-quincy
Yuri Weinstein [Tue, 1 Oct 2024 15:05:34 +0000 (08:05 -0700)]
Merge pull request #59370 from NitzanMordhai/wip-67643-quincy

quincy: mgr/rest: Trim  requests array and limit size

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
9 months agoceph.spec.in: we need jsonnet for all distroes for make check 60074/head
Kyr Shatskyy [Thu, 18 Jul 2024 00:45:26 +0000 (02:45 +0200)]
ceph.spec.in: we need jsonnet for all distroes for make check

Fixes: https://tracker.ceph.com/issues/67938
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
(cherry picked from commit e7139a8f438c2a019de4c1fc8a81d735ca27e35a)

9 months agomds: invalid id for client eviction is to be treated as success 58724/head
Venky Shankar [Thu, 19 Sep 2024 06:27:53 +0000 (06:27 +0000)]
mds: invalid id for client eviction is to be treated as success

Introduced-by: 0ef5941a2e79
Fixes: http://tracker.ceph.com/issues/68132
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4277cd298977c606609b90e2c2fa367cce97ae5a)

9 months agoMerge pull request #59109 from YiteGu/wip-61396-quincy
SrinivasaBharathKanta [Mon, 30 Sep 2024 23:55:00 +0000 (05:25 +0530)]
Merge pull request #59109 from YiteGu/wip-61396-quincy

quincy: osd/TrackedOp: Fix TrackedOp event order

9 months agocommon/Formatter: dump inf/nan as null 60064/head
sajibreadd [Fri, 24 May 2024 11:27:26 +0000 (17:27 +0600)]
common/Formatter: dump inf/nan as null

Fixes: https://tracker.ceph.com/issues/66215
Signed-off-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
(cherry picked from commit 214328d9daa70a9299255bf07e7169dac1ccf5bd)

Conflicts:
src/common/Formatter.h

9 months agomds: do remove the cap when seqs equal or larger than last issue 58296/head
Xiubo Li [Thu, 11 Apr 2024 01:53:04 +0000 (09:53 +0800)]
mds: do remove the cap when seqs equal or larger than last issue

There is a race in case of:

   MDS                            rw Client
- Issue the 'Asx' caps to
  rw client
                             - Adds the cap, then removes it
       later by queuing it to the cap
       release list. But the cap->seq
       may have been updated by previous
       cap grant requests.
       And the cap grant request won't
       increase the 'last_issue' seq in
       MDS.
- ro client's lookup
  request comes and the
  MDS sends a 'Ax' caps
  revoke request to rw
  client by increasing
  the 'seq'.
                             - The revoke request just finds
       that the cap doesn't exist, then
       queues a new cap release
       immediately with the new 'seq'.
       Then trigger to flush the pending
       cap releases to MDS.
- Just receives the cap
  release request but the
  'seq' > cap's 'last_issue',
  then MDS will skip
  removing the cap. And
  then the _do_cap_release()
  will issue the 'Ax' caps
  back to rw client.

  Then wakes up the ro
  client's lookup request,
  while the lookup request
  will try to revoke the
  'Ax' caps again from the
  rw client.

This will cause a spinlock infinitely in mds side.

Fixes: https://tracker.ceph.com/issues/64977
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 345978e7607e227854ccc78b066b274b97940391)

9 months agoMerge pull request #60048 from zdover23/wip-doc-2024-09-30-backport-60039-to-quincy
Anthony D'Atri [Mon, 30 Sep 2024 04:37:21 +0000 (00:37 -0400)]
Merge pull request #60048 from zdover23/wip-doc-2024-09-30-backport-60039-to-quincy

quincy: doc/rados: edit "Placement Groups Never Get Clean"

9 months agodoc/rados: edit "Placement Groups Never Get Clean" 60048/head
Zac Dover [Sun, 29 Sep 2024 06:10:29 +0000 (16:10 +1000)]
doc/rados: edit "Placement Groups Never Get Clean"

Make grammar improvements (and correct a verb disagreement) in the
section "Placement Groups Never Get Clean" in
doc/rados/troubleshooting/troubleshooting-pg.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2baa027b13c0b4ac8b9fb864858a42e010ca8948)

9 months agoMerge pull request #58871 from kshtsk/wip-quincy-containerized-make-check
kyr [Sun, 29 Sep 2024 15:13:38 +0000 (17:13 +0200)]
Merge pull request #58871 from kshtsk/wip-quincy-containerized-make-check

quincy: run-make-check: use get_processors in run-make-check script

9 months agoMerge pull request #60035 from zdover23/wip-doc-2024-09-28-backport-60030-to-quincy
Anthony D'Atri [Sat, 28 Sep 2024 03:18:55 +0000 (23:18 -0400)]
Merge pull request #60035 from zdover23/wip-doc-2024-09-28-backport-60030-to-quincy

quincy: doc: Document the Windows CI job

9 months agodoc: Document the Windows CI job 60035/head
Lucian Petrut [Fri, 27 Sep 2024 14:09:34 +0000 (14:09 +0000)]
doc: Document the Windows CI job

We're adding a document that describes:

* the steps performed by the Windows CI job
* artifact structure
* frequently asked questions

The document is meant to assist the Ceph developers in investigating
CI failures. This is especially important as the Windows CI job runs
integration tests that would otherwise only be executed by
Teuthology, thus helping catch potential regressions quickly.

Note that the identified regressions are not necessarily Windows
specific, usually affecting Linux builds as well.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit bb7d5b62d9db13ca8c0268d00b8e353443285957)

9 months agoMerge pull request #59317 from k0ste/wip-57417-quincy
Shilpa Jagannath [Thu, 26 Sep 2024 17:33:37 +0000 (10:33 -0700)]
Merge pull request #59317 from k0ste/wip-57417-quincy

quincy: rgw: Get canonical storage class when storage class is empty in

9 months agoMerge pull request #59312 from k0ste/wip-54151-quincy
Shilpa Jagannath [Thu, 26 Sep 2024 17:33:23 +0000 (10:33 -0700)]
Merge pull request #59312 from k0ste/wip-54151-quincy

quincy: rgwlc: fix compat-decoding of cls_rgw_lc_get_entry_ret

9 months agoMerge pull request #59993 from zdover23/wip-doc-2024-09-26-backport-59986-to-quincy
Anthony D'Atri [Thu, 26 Sep 2024 11:35:52 +0000 (07:35 -0400)]
Merge pull request #59993 from zdover23/wip-doc-2024-09-26-backport-59986-to-quincy

quincy: doc: fix typo

9 months agodoc: fix typo 59993/head
N Balachandran [Thu, 26 Sep 2024 04:16:56 +0000 (09:46 +0530)]
doc: fix typo

Fixed a typo in the jaeger traces documentation.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 5a11bc00e6c068c965bb7d961feeb47353cc893c)

9 months agoMerge pull request #59909 from zdover23/wip-doc-2024-09-21-backport-59904-to-quincy
Anthony D'Atri [Fri, 20 Sep 2024 17:06:13 +0000 (13:06 -0400)]
Merge pull request #59909 from zdover23/wip-doc-2024-09-21-backport-59904-to-quincy

quincy: doc: Update "Getting Started" to link to start not install

9 months agodoc: Update "Getting Started" to link to start not install 59909/head
Matthew Vernon [Fri, 20 Sep 2024 14:07:37 +0000 (15:07 +0100)]
doc: Update "Getting Started" to link to start not install

Currently the "Getting Started" link points at the install page, not the "Intro to Ceph" section, which seems more appropriate.

If you're happy with this, can it get backported to the still-supported releases, please?

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit d257eacb05a463c93872ae8aa8210806affda4ec)

9 months agoMerge pull request #59873 from zdover23/wip-doc-2024-09-19-backport-59856-to-quincy
Anthony D'Atri [Thu, 19 Sep 2024 11:40:52 +0000 (07:40 -0400)]
Merge pull request #59873 from zdover23/wip-doc-2024-09-19-backport-59856-to-quincy

quincy: doc/rados: add confval directives to health-checks

9 months agoWarning added for slow operations and stalled read in BlueStore. User can control... 59468/head
sajibreadd [Mon, 27 May 2024 07:30:06 +0000 (13:30 +0600)]
Warning added for slow operations and stalled read in BlueStore. User can control how much time the warning should persist after last occurence and maximum number of operations as a threshold will be considered for the warning.

Fixes: https://tracker.ceph.com/issues/62500
Signed-off-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>
(cherry picked from commit 73b80a9a2c38259346fb646f85fa2ba4dcbb1329)

Conflicts:
src/blk/BlockDevice.h
src/common/options/global.yaml.in
src/os/bluestore/BlueStore.cc
src/os/bluestore/BlueStore.h
        doc/rados/operations/health-checks.rst

9 months agoMerge pull request #59500 from idryomov/wip-67698-quincy
Ilya Dryomov [Thu, 19 Sep 2024 07:21:01 +0000 (09:21 +0200)]
Merge pull request #59500 from idryomov/wip-67698-quincy

quincy: rbd: "rbd bench" always writes the same byte

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agoMerge pull request #59659 from idryomov/wip-67845-quincy
Ilya Dryomov [Thu, 19 Sep 2024 07:19:40 +0000 (09:19 +0200)]
Merge pull request #59659 from idryomov/wip-67845-quincy

quincy: librbd/migration: prune snapshot extents in RawFormat::list_snaps()

Reviewed-by: Ramana Raja <rraja@redhat.com>