]> git.apps.os.sepia.ceph.com Git - ceph-ci.git/log
ceph-ci.git
2 years agomgr/dashboard: fix bootstrap script for cephadm installation
avanthakkar [Wed, 13 Sep 2023 10:49:01 +0000 (16:19 +0530)]
mgr/dashboard: fix bootstrap script for cephadm installation

Fixes: https://tracker.ceph.com/issues/62827
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2 years agoMerge pull request #53791 from zdover23/wip-doc-2023-10-03-rados-operations-control...
zdover23 [Wed, 4 Oct 2023 09:30:23 +0000 (20:30 +1100)]
Merge pull request #53791 from zdover23/wip-doc-2023-10-03-rados-operations-control-1-of-x

doc/rados: edit ops/control.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit ops/control.rst (1 of x)
Zac Dover [Tue, 3 Oct 2023 12:22:39 +0000 (22:22 +1000)]
doc/rados: edit ops/control.rst (1 of x)

Edit doc/rados/operations/control.rst (1 of x).

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge PR #53494 into main
Venky Shankar [Wed, 4 Oct 2023 08:18:31 +0000 (13:48 +0530)]
Merge PR #53494 into main

* refs/pull/53494/head:
mds: fix stray CInodes' use-after-free bug when submit ELid entry

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #53332 from zhsgao/mds_overload_epochs
Venky Shankar [Wed, 4 Oct 2023 08:16:27 +0000 (13:46 +0530)]
Merge pull request #53332 from zhsgao/mds_overload_epochs

mds: add option mds_bal_overload_epochs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #53746 from rhcs-dashboard/enhance-usage-bar
Pedro Gonzalez Gomez [Wed, 4 Oct 2023 07:37:28 +0000 (09:37 +0200)]
Merge pull request #53746 from rhcs-dashboard/enhance-usage-bar

mgr/dashboard: rbd image hide usage bar when disk usage is not provided

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #53307 from rhcs-dashboard/fix-62721-main
Nizamudeen A [Wed, 4 Oct 2023 05:00:05 +0000 (10:30 +0530)]
Merge pull request #53307 from rhcs-dashboard/fix-62721-main

mgr/dashboard: Rgw Multi-site naming improvements

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #52938 from batrick/mon-loop-refactor
Yuri Weinstein [Tue, 3 Oct 2023 19:58:15 +0000 (12:58 -0700)]
Merge pull request #52938 from batrick/mon-loop-refactor

mon: refactor loop variable names

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #52733 from batrick/i59813
Yuri Weinstein [Tue, 3 Oct 2023 19:57:42 +0000 (12:57 -0700)]
Merge pull request #52733 from batrick/i59813

osd/OSDMonitor: check svc is writeable before changing pending

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #53449 from ljflores/wip-tracker-62833
Laura Flores [Tue, 3 Oct 2023 19:53:21 +0000 (15:53 -0400)]
Merge pull request #53449 from ljflores/wip-tracker-62833

osd: fix read balancer logic to avoid redundant primary assignment

2 years agoMerge pull request #53671 from rhcs-dashboard/fix-protect-if-layering-enabled
Nizamudeen A [Tue, 3 Oct 2023 12:57:24 +0000 (18:27 +0530)]
Merge pull request #53671 from rhcs-dashboard/fix-protect-if-layering-enabled

mgr/dashboard: enable protect option if layering enabled

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>
2 years agoMerge pull request #53673 from rhcs-dashboard/fix-62984-main
Nizamudeen A [Tue, 3 Oct 2023 12:54:09 +0000 (18:24 +0530)]
Merge pull request #53673 from rhcs-dashboard/fix-62984-main

mgr/dashboard: show a message to restart the rgw daemons after moving from single-site to multi-site

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #53703 from adk3798/cephadm-refactor-ssh
Adam King [Tue, 3 Oct 2023 12:46:30 +0000 (08:46 -0400)]
Merge pull request #53703 from adk3798/cephadm-refactor-ssh

cephadm: start ssh.py in cephadmlib

Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge PR #53600 into main
Patrick Donnelly [Tue, 3 Oct 2023 12:30:20 +0000 (12:30 +0000)]
Merge PR #53600 into main

* refs/pull/53600/head:
mds/FSMap: allow upgrades if no up mds

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agomgr/dashboard: enable protect option if layering enabled
avanthakkar [Tue, 26 Sep 2023 11:04:08 +0000 (16:34 +0530)]
mgr/dashboard: enable protect option if layering enabled

Fixes: https://tracker.ceph.com/issues/63076
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2 years agomgr/dashboard: show a message to restart the rgw daemons after moving from single...
Aashish Sharma [Tue, 26 Sep 2023 12:27:39 +0000 (17:57 +0530)]
mgr/dashboard: show a message to restart the rgw daemons after moving from single-site to multi-site

Fixes: https://tracker.ceph.com/issues/62984
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #53522 from rhcs-dashboard/landing-page-layout-fix
Nizamudeen A [Tue, 3 Oct 2023 09:59:45 +0000 (15:29 +0530)]
Merge pull request #53522 from rhcs-dashboard/landing-page-layout-fix

mgr/dashboard: fix the landing page layout issues

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agomgr/dashboard: rbd image hide usage bar when disk usage is not provided
Pedro Gonzalez Gomez [Mon, 2 Oct 2023 08:35:08 +0000 (10:35 +0200)]
mgr/dashboard: rbd image hide usage bar when disk usage is not provided

Fixes: https://tracker.ceph.com/issues/63037
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2 years agoMerge pull request #53699 from rhcs-dashboard/tls_v1.2_toglle
Nizamudeen A [Tue, 3 Oct 2023 07:52:23 +0000 (13:22 +0530)]
Merge pull request #53699 from rhcs-dashboard/tls_v1.2_toglle

mgr/dashboard: allow tls 1.2 with a config option

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #53697 from rhcs-dashboard/mds-number-validator-fix
Nizamudeen A [Tue, 3 Oct 2023 07:50:39 +0000 (13:20 +0530)]
Merge pull request #53697 from rhcs-dashboard/mds-number-validator-fix

mgr/dashboard: fix cephfs form validator

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2 years agomgr/dashboard: fix the landing page layout issues
Nizamudeen A [Tue, 19 Sep 2023 06:42:36 +0000 (12:12 +0530)]
mgr/dashboard: fix the landing page layout issues

We were following a row-col grid layout for the landing page.
First row includes Details, Status and Capacity
Second row for Inventory and Cluster Utilization

So if one of the item in the first row increases, it pushes the entire
second row downwards.

To fix this, I made a col-row grid.

First col has Details and Inventory in two rows.
Second col has Status and Capacity as a col and Cluster Utilization as a
single row

Fixes: https://tracker.ceph.com/issues/62961
Signed-off-by: Nizamudeen A <nia@redhat.com>
Co-authored-by: cloudbehl <cloudbehl@gmail.com>
2 years agoMerge PR #53754 into main
Patrick Donnelly [Tue, 3 Oct 2023 02:39:09 +0000 (02:39 +0000)]
Merge PR #53754 into main

* refs/pull/53754/head:
osd: correct unsigned/signed compiler wrn

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2 years agoosd: correct unsigned/signed compiler wrn
Patrick Donnelly [Mon, 2 Oct 2023 16:42:28 +0000 (16:42 +0000)]
osd: correct unsigned/signed compiler wrn

    /home/pdonnell/ceph/src/osd/OSD.cc: In member function ‘void OSD::ShardedOpWQ::stop_for_fast_shutdown()’:
    /home/pdonnell/ceph/src/osd/OSD.cc:11143:41: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
    11143 |   for (int shard_index = 0; shard_index < osd->num_shards; shard_index++) {

Fixes: https://tracker.ceph.com/issues/62851
Fixes: 210dbd4ff19ea66fd2f0109cc15aad53349be52f
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #47148 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks...
Casey Bodley [Mon, 2 Oct 2023 19:46:29 +0000 (20:46 +0100)]
Merge pull request #47148 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks-start

test/cls_lock: expired lock before unlock and start check

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #53616 from ronen-fr/wip-rf-cnceq
Casey Bodley [Mon, 2 Oct 2023 19:29:57 +0000 (20:29 +0100)]
Merge pull request #53616 from ronen-fr/wip-rf-cnceq

rgw/test: fix compiler warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agomds/FSMap: allow upgrades if no up mds
Patrick Donnelly [Fri, 22 Sep 2023 18:05:23 +0000 (14:05 -0400)]
mds/FSMap: allow upgrades if no up mds

This is to support the fail_fs scenario for cephadm where max_mds >= 1
and all MDS are down.

Fixes: https://tracker.ceph.com/issues/62682
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocephadm: format black cephadmlib/ssh.py
Adam King [Wed, 27 Sep 2023 19:51:14 +0000 (15:51 -0400)]
cephadm: format black cephadmlib/ssh.py

Signed-off-by: Adam King <adking@redhat.com>
2 years agocephadm: start ssh.py in cephadmlib
Adam King [Wed, 27 Sep 2023 19:49:37 +0000 (15:49 -0400)]
cephadm: start ssh.py in cephadmlib

As part of the cephadm refactoring process
to split cephadm into multiple python files,
start "ssh.py" that includes some functions used
for setting up and testing ssh connections,
primarily as part of bootstrap.

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #53735 from idryomov/wip-63010
Ilya Dryomov [Mon, 2 Oct 2023 15:10:19 +0000 (17:10 +0200)]
Merge pull request #53735 from idryomov/wip-63010

qa/suites/krbd: stress test for recovering from watch errors

Reviewed-by: Mykola Golub <mgolub@suse.com>
2 years agoMerge pull request #50350 from BBoozmen/oozmen_rgw_beast_log_bucket_name
Casey Bodley [Mon, 2 Oct 2023 14:39:11 +0000 (15:39 +0100)]
Merge pull request #50350 from BBoozmen/oozmen_rgw_beast_log_bucket_name

rgw/logging: add bucket and object names to the log line for ops-log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52144 from linuxbox2/wip-nochown-bucket
Casey Bodley [Mon, 2 Oct 2023 14:38:35 +0000 (15:38 +0100)]
Merge pull request #52144 from linuxbox2/wip-nochown-bucket

rgw/file: make setattr(...) a no-op on buckets

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #52960 from cbodley/wip-62411
Casey Bodley [Mon, 2 Oct 2023 14:38:16 +0000 (15:38 +0100)]
Merge pull request #52960 from cbodley/wip-62411

rgw/lc: remove_bucket_config() doesn't update xattrs on bucket delete

Reviewed-by Matt Benjamin <mbenjamin@redhat.com>

2 years agoMerge pull request #53266 from VVoidV/lbr-chunk-upload-buffer-fix
Casey Bodley [Mon, 2 Oct 2023 14:37:46 +0000 (15:37 +0100)]
Merge pull request #53266 from VVoidV/lbr-chunk-upload-buffer-fix

rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #53525 from RickyMaRui/fix-SignatureDoesNotMatch
Casey Bodley [Mon, 2 Oct 2023 14:37:25 +0000 (15:37 +0100)]
Merge pull request #53525 from RickyMaRui/fix-SignatureDoesNotMatch

rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #53602 from cbodley/wip-62938
Casey Bodley [Mon, 2 Oct 2023 14:37:08 +0000 (15:37 +0100)]
Merge pull request #53602 from cbodley/wip-62938

rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #53614 from xrmeng8756/master
Casey Bodley [Mon, 2 Oct 2023 14:36:53 +0000 (15:36 +0100)]
Merge pull request #53614 from xrmeng8756/master

rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…

Reviewed-by: Jiffin Tony Thottan <jthottan@redhat.com>
2 years agoMerge pull request #53676 from smanjara/data-sync-fairness-crash
Casey Bodley [Mon, 2 Oct 2023 14:36:07 +0000 (15:36 +0100)]
Merge pull request #53676 from smanjara/data-sync-fairness-crash

rgw/multisite: call drain before flushing markers in incremental sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #53680 from cbodley/wip-62989
Casey Bodley [Mon, 2 Oct 2023 14:35:47 +0000 (15:35 +0100)]
Merge pull request #53680 from cbodley/wip-62989

rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #53684 from cbodley/wip-rgw-keystone-yield
Casey Bodley [Mon, 2 Oct 2023 14:35:20 +0000 (15:35 +0100)]
Merge pull request #53684 from cbodley/wip-rgw-keystone-yield

rgw/async: use optional_yield for keystone and kms requests

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2 years agoMerge pull request #53691 from max-licc/radosgw_start_watcher_crash
Casey Bodley [Mon, 2 Oct 2023 14:34:51 +0000 (15:34 +0100)]
Merge pull request #53691 from max-licc/radosgw_start_watcher_crash

rgw: fix unwatch crash at radosgw startup

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2 years agoMerge pull request #53714 from cbodley/wip-63004
Casey Bodley [Mon, 2 Oct 2023 14:34:25 +0000 (15:34 +0100)]
Merge pull request #53714 from cbodley/wip-63004

[CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #53724 from adk3798/rm-get_unit_name_by_instance
Adam King [Mon, 2 Oct 2023 13:46:04 +0000 (09:46 -0400)]
Merge pull request #53724 from adk3798/rm-get_unit_name_by_instance

cephadm: remove get_unit_name_by_instance func

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #53715 from adk3798/other-haproxy-container-version
Adam King [Mon, 2 Oct 2023 13:41:38 +0000 (09:41 -0400)]
Merge pull request #53715 from adk3798/other-haproxy-container-version

cephadm: fix haproxy version with certain containers

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #53744 from zdover23/wip-doc-2023-10-02-architecture-16-of-x
zdover23 [Mon, 2 Oct 2023 12:01:54 +0000 (23:01 +1100)]
Merge pull request #53744 from zdover23/wip-doc-2023-10-02-architecture-16-of-x

doc/architecture: edit "Calculating PG IDs"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoqa/suites/krbd: stress test for recovering from watch errors
Ilya Dryomov [Wed, 27 Sep 2023 18:17:11 +0000 (20:17 +0200)]
qa/suites/krbd: stress test for recovering from watch errors

Fixes: https://tracker.ceph.com/issues/63010
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #53606 from Patty8122/control-reaches-end-fix
Ronen Friedman [Mon, 2 Oct 2023 05:37:18 +0000 (08:37 +0300)]
Merge pull request #53606 from Patty8122/control-reaches-end-fix

test: corrected control reaches end by adding a return

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2 years agodoc/architecture: edit "Calculating PG IDs"
Zac Dover [Sun, 1 Oct 2023 23:43:37 +0000 (09:43 +1000)]
doc/architecture: edit "Calculating PG IDs"

Edit the section "Calcluating PG IDs" in doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agotest: corrected control reaches end by adding a return
Patty8122 [Fri, 22 Sep 2023 21:10:13 +0000 (16:10 -0500)]
test: corrected control reaches end by adding a return

Signed-off-by: Patty8122 <divyapattisapu@uchicago.edu>
2 years agoMerge pull request #53580 from AliMasarweh/wip-alimasa-persistent-q-migration
Ali Masarwa [Sun, 1 Oct 2023 13:38:40 +0000 (16:38 +0300)]
Merge pull request #53580 from AliMasarweh/wip-alimasa-persistent-q-migration

RGW | Bucket Notification: migrating old entries to support persistency control

Reviewed-by: yuvalif <ylifshit@redhat.com>
2 years agoMerge pull request #46692 from pdvian/wip-unreachable-osd
Yuri Weinstein [Sun, 1 Oct 2023 12:59:26 +0000 (05:59 -0700)]
Merge pull request #46692 from pdvian/wip-unreachable-osd

osd: Report health error if OSD public address is not within subnet

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #52509 from ericqzhao/main
Yuri Weinstein [Sun, 1 Oct 2023 12:57:01 +0000 (05:57 -0700)]
Merge pull request #52509 from ericqzhao/main

mgr: add throttle policy for DaemonServer

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #53740 from zdover23/wip-doc-2023-09-30-architecture-15-of-x
zdover23 [Sun, 1 Oct 2023 11:50:24 +0000 (22:50 +1100)]
Merge pull request #53740 from zdover23/wip-doc-2023-09-30-architecture-15-of-x

doc/architecture: edit several sections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #53618 from adk3798/cephadm-refactor-more-net-utils
Adam King [Sat, 30 Sep 2023 22:41:29 +0000 (18:41 -0400)]
Merge pull request #53618 from adk3798/cephadm-refactor-more-net-utils

cephadm: move more funcs into net_utils.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agodoc/architecture: edit several sections
Zac Dover [Sat, 30 Sep 2023 04:58:41 +0000 (14:58 +1000)]
doc/architecture: edit several sections

Edit the following sections in doc/architecture.rst:

 1. Dynamic Cluster Management
 2. About Pools
 3. Mapping PGs to OSDs

The tone of "Dynamic Cluster Management" remains a bit too close to the
tone of marketing material, in my opinion, but I will return to firm it
up when I have finished a once-over of architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #53702 from adk3798/cephadm-refactor-registry-login
Adam King [Sat, 30 Sep 2023 21:07:45 +0000 (17:07 -0400)]
Merge pull request #53702 from adk3798/cephadm-refactor-registry-login

cephadm: move registry_login to container_engines.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #53617 from adk3798/cephadm-refactor-more-data-utils
Adam King [Sat, 30 Sep 2023 21:05:47 +0000 (17:05 -0400)]
Merge pull request #53617 from adk3798/cephadm-refactor-more-data-utils

cephadm: move more funcs into data_utils.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #53612 from adk3798/fix-refreshed-column
Adam King [Sat, 30 Sep 2023 20:59:42 +0000 (16:59 -0400)]
Merge pull request #53612 from adk3798/fix-refreshed-column

mgr/cephadm: fix REFRESHED column of orch ps being unpopulated

Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agocephadm: re-format black data_utils.py
Adam King [Sat, 23 Sep 2023 16:16:06 +0000 (12:16 -0400)]
cephadm: re-format black data_utils.py

Signed-off-by: Adam King <adking@redhat.com>
2 years agocephadm: move more funcs into data_utils.py
Adam King [Sat, 23 Sep 2023 16:15:06 +0000 (12:15 -0400)]
cephadm: move more funcs into data_utils.py

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #53682 from phlogistonjohn/jjm-cephadm-move-pathify
Adam King [Sat, 30 Sep 2023 19:25:59 +0000 (15:25 -0400)]
Merge pull request #53682 from phlogistonjohn/jjm-cephadm-move-pathify

cephadm: move pathify & get_file_timestamp to file_utils

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #53683 from phlogistonjohn/jjm-cephadm-move-getters
Adam King [Sat, 30 Sep 2023 19:22:12 +0000 (15:22 -0400)]
Merge pull request #53683 from phlogistonjohn/jjm-cephadm-move-getters

cephadm: move context getters

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #53726 from zdover23/wip-doc-2023-09-29-architecture-14-of-x
zdover23 [Sat, 30 Sep 2023 00:12:15 +0000 (10:12 +1000)]
Merge pull request #53726 from zdover23/wip-doc-2023-09-29-architecture-14-of-x

doc/architecture: edit "Replication"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph.@gmail.com>
2 years agoMerge pull request #53605 from sayantani11/osd-1
Laura Flores [Fri, 29 Sep 2023 22:34:45 +0000 (18:34 -0400)]
Merge pull request #53605 from sayantani11/osd-1

script/backport-resolve-issue: Update script with latest versions

2 years agorgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_read_bytes and...
mengxiangrui [Sat, 23 Sep 2023 02:06:15 +0000 (10:06 +0800)]
rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_read_bytes and max_write_bytes field mismatch

Fixes: https://tracker.ceph.com/issues/62955
Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
2 years agoMerge pull request #53723 from zdover23/wip-doc-2023-09-29-architecture-13-of-x
zdover23 [Fri, 29 Sep 2023 06:02:14 +0000 (16:02 +1000)]
Merge pull request #53723 from zdover23/wip-doc-2023-09-29-architecture-13-of-x

doc/architecture: edit "Data Scrubbing"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/architecture: edit "Replication"
Zac Dover [Thu, 28 Sep 2023 21:37:40 +0000 (07:37 +1000)]
doc/architecture: edit "Replication"

Edit "Replication" in the "Smart Daemons Enable Hyperscale" section of
doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agodoc/architecture: edit "Data Scrubbing"
Zac Dover [Thu, 28 Sep 2023 18:40:08 +0000 (04:40 +1000)]
doc/architecture: edit "Data Scrubbing"

Edit the "Data Scrubbing" listitem in the list of benefits conferred by
the use by OSDs of the aggregate power of the cluster, in the section
"Smart Daemons Enable Hyperscale" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge PR #53594 into main
Venky Shankar [Fri, 29 Sep 2023 04:31:17 +0000 (10:01 +0530)]
Merge PR #53594 into main

* refs/pull/53594/head:
qa: fix "no orch backend set" in nfs suite

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #53707 from zdover23/wip-doc-2023-09-28-architecture-12-of-x
zdover23 [Thu, 28 Sep 2023 21:35:45 +0000 (07:35 +1000)]
Merge pull request #53707 from zdover23/wip-doc-2023-09-28-architecture-12-of-x

doc/architecture: edit "OSD Membership and Status"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agocephadm: remove get_unit_name_by_instance func
Adam King [Thu, 28 Sep 2023 18:39:53 +0000 (14:39 -0400)]
cephadm: remove get_unit_name_by_instance func

As it is one line, quite simple, and only
had a single caller, it was decided we'd remove
this function as part of the cephadm refactor.

Signed-off-by: Adam King <adking@redhat.com>
2 years agodoc/architecture: edit "OSD Membership and Status"
Zac Dover [Thu, 28 Sep 2023 03:55:02 +0000 (13:55 +1000)]
doc/architecture: edit "OSD Membership and Status"

Edit "OSD Membership and Status" in the "Smart Daemons Enable
Hyperscale" section of doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agocephadm: fix haproxy version with certain containers
Adam King [Thu, 28 Sep 2023 15:07:21 +0000 (11:07 -0400)]
cephadm: fix haproxy version with certain containers

Some builds of haproxy containers' output
from "haproxy -v" start with

HAProxy version

rather than

HA-Proxy version

no reason on our end not to accept both

Signed-off-by: Adam King <adking@redhat.com>
2 years agorgw: Fix bucket validation against POST policies
Joshua Baergen [Wed, 17 May 2023 18:17:09 +0000 (12:17 -0600)]
rgw: Fix bucket validation against POST policies

It's possible that user could provide a form part as a part of a POST
object upload that uses 'bucket' as a key; in this case, it was
overriding what was being set in the validation env (which is the real
bucket being modified). The result of this is that a user could actually
upload to any bucket accessible by the specified access key by matching
the bucket in the POST policy in said POST form part.

Fix this simply by setting the bucket to the correct value after the
POST form parts are processed, ignoring the form part above if
specified.

Fixes: https://tracker.ceph.com/issues/63004
Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
2 years agoMerge pull request #53608 from cbodley/wip-62536
Casey Bodley [Thu, 28 Sep 2023 14:37:20 +0000 (15:37 +0100)]
Merge pull request #53608 from cbodley/wip-62536

qa/rgw/sts: keycloak task installs java manually

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agorgw/file: make setattr(...) a no-op on buckets
Matt Benjamin [Tue, 20 Jun 2023 19:31:26 +0000 (15:31 -0400)]
rgw/file: make setattr(...) a no-op on buckets

Shallow fix for apparent unstable behavior after nfs "chown" on
an RGW bucket via RGW NFS.  While we allow buckets to be created
(and subject to ordinary rules, deleted), chown against a bucket
hasn't been tested and potentially is not valid.  Prevent it
altogether for now--if permissions would allow it, chown will
succeed but won't have any effect.

Fixes: https://tracker.ceph.com/issues/61689
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoRGW | Bucket Notification: migrating old entries to support persistency control
Ali Masarwa [Tue, 19 Sep 2023 11:32:42 +0000 (14:32 +0300)]
RGW | Bucket Notification: migrating old entries to support persistency control

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2 years agoqa/suites/krbd: rename singleton to singleton-msgr-failures
Ilya Dryomov [Thu, 28 Sep 2023 09:52:06 +0000 (11:52 +0200)]
qa/suites/krbd: rename singleton to singleton-msgr-failures

A "singleton without msgr-failures" is wanted in the next commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #53432 from liu-chunmei/seastore_device_size
Liu-Chunmei [Thu, 28 Sep 2023 03:11:47 +0000 (20:11 -0700)]
Merge pull request #53432 from liu-chunmei/seastore_device_size

crimson/vstart: default seastore_device_size will be out of space f…

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocephadm: re-format black container_engines.py
Adam King [Wed, 27 Sep 2023 19:24:43 +0000 (15:24 -0400)]
cephadm: re-format black container_engines.py

Signed-off-by: Adam King <adking@redhat.com>
2 years agocephadm: move registry_login to container_engines.py
Adam King [Wed, 27 Sep 2023 18:34:42 +0000 (14:34 -0400)]
cephadm: move registry_login to container_engines.py

Signed-off-by: Adam King <adking@redhat.com>
2 years agocephadm: move logging from registry_login to command_registry_login
Adam King [Wed, 27 Sep 2023 18:27:18 +0000 (14:27 -0400)]
cephadm: move logging from registry_login to command_registry_login

So that registry_login can be moved to container_engines.py
without creating a dependency on logging there

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #53607 from cfsnyder/wip-cfsnyder-bucket-check-bug
Casey Bodley [Wed, 27 Sep 2023 17:38:42 +0000 (18:38 +0100)]
Merge pull request #53607 from cfsnyder/wip-cfsnyder-bucket-check-bug

rgw: fix radosgw-admin bucket check stat calculation bug

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52393 from yuvalif/wip-yuval-enable-trace-lua
Yuval Lifshitz [Wed, 27 Sep 2023 15:07:28 +0000 (18:07 +0300)]
Merge pull request #52393 from yuvalif/wip-yuval-enable-trace-lua

common/tracer: remove is_enabled check in add_span methods

reviewed-by: cbodley, ljflores

2 years agomgr/dashboard: allow tls 1.2 with a config option
Nizamudeen A [Wed, 27 Sep 2023 11:27:32 +0000 (16:57 +0530)]
mgr/dashboard: allow tls 1.2 with a config option

Provide the option to allow tls1.2

`ceph dashboard set-enable-unsafe-tls-v1-2 True` followed with a mgr
restart will enable tls 1.2.

With tls1.2 enabled
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:56 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00018s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
|_http-server-header: Ceph-Dashboard
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 16.55 seconds
```

Without tls1.2 enabled (which defaults to tls 1.3)
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:54 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000075s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
| ssl-enum-ciphers:
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A
|_http-server-header: Ceph-Dashboard
```

Fixes: https://tracker.ceph.com/issues/62940
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agorgw: fix unwatch crash at radosgw startup
lichaochao [Wed, 27 Sep 2023 10:06:34 +0000 (12:06 +0200)]
rgw: fix unwatch crash at radosgw startup

During radosgw initialization, if there is an exception in init_watch that causes the watcher registration to fail,
When finalize_watch is executed, a crash occurs due to unregister an unregistered watch.

Fixes: https://tracker.ceph.com/issues/60094
Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
2 years agomgr/dashboard: fix cephfs form validator
Nizamudeen A [Wed, 27 Sep 2023 09:43:34 +0000 (15:13 +0530)]
mgr/dashboard: fix cephfs form validator

Number is not allowed as the starting character of the mds service

Fixes: https://tracker.ceph.com/issues/63005
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #53523 from rhcs-dashboard/add-ceph-daemon-label-labeled-counters
Ilya Dryomov [Wed, 27 Sep 2023 07:27:04 +0000 (09:27 +0200)]
Merge pull request #53523 from rhcs-dashboard/add-ceph-daemon-label-labeled-counters

exporter: add ceph_daemon labels to labeled counters as well

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #52828 from rhcs-dashboard/angular-15
Nizamudeen A [Wed, 27 Sep 2023 05:41:35 +0000 (11:11 +0530)]
Merge pull request #52828 from rhcs-dashboard/angular-15

mgr/dashboard: upgrade to angular v15

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2 years agoMerge pull request #53596 from rhcs-dashboard/fix-validations-in-cephfs-forms
Nizamudeen A [Wed, 27 Sep 2023 05:13:14 +0000 (10:43 +0530)]
Merge pull request #53596 from rhcs-dashboard/fix-validations-in-cephfs-forms

mgr/dashboard: fix cephfs forms validations

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2 years agoMerge PR #53529 into main
Venky Shankar [Wed, 27 Sep 2023 04:27:25 +0000 (09:57 +0530)]
Merge PR #53529 into main

* refs/pull/53529/head:
doc: update colorama, packaging
cephfs-shell: drop LooseVersion for version.parse

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #53685 from zdover23/wip-doc-2023-09-27-architecture-11-of-x
zdover23 [Tue, 26 Sep 2023 20:54:06 +0000 (06:54 +1000)]
Merge pull request #53685 from zdover23/wip-doc-2023-09-27-architecture-11-of-x

doc/architecture: edit "OSDs service clients directly"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/architecture: edit "OSDs service clients directly"
Zac Dover [Tue, 26 Sep 2023 20:00:34 +0000 (06:00 +1000)]
doc/architecture: edit "OSDs service clients directly"

Edit "OSDs service clients directly" in the list in
"Smart Daemons Enable Hyperscale" in doc/architecure.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agorgw/async: use optional_yield for keystone and kms requests
Casey Bodley [Tue, 26 Sep 2023 17:46:20 +0000 (13:46 -0400)]
rgw/async: use optional_yield for keystone and kms requests

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agocephadm: remove (doc)string
John Mulligan [Tue, 26 Sep 2023 17:45:35 +0000 (13:45 -0400)]
cephadm: remove (doc)string

Remove a, now irrelevant (IMO), docstring that might have been
associated with the recently moved `cached_stdin` global. It's not
really clear how helpful it is in light of the new "compiled"
cephadm, so I am opting to remove it rather than move it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agocephadm: black format context_getters.py
John Mulligan [Tue, 26 Sep 2023 17:29:55 +0000 (13:29 -0400)]
cephadm: black format context_getters.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agocephadm: rename fetch_tcp_ports to fetch_endpoints
John Mulligan [Tue, 26 Sep 2023 17:28:44 +0000 (13:28 -0400)]
cephadm: rename fetch_tcp_ports to fetch_endpoints

Rename fetch_tcp_ports to fetch_endpoints to more closely match what
the function is doing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agocephadm: move context based getters to context_getters.py
John Mulligan [Tue, 26 Sep 2023 17:26:23 +0000 (13:26 -0400)]
cephadm: move context based getters to context_getters.py

Move functions that exist mainly to pull information out of the
CephadmContext in various ways to a new context_getters.py module.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agocephadm: move a logging line closer to where the data is used
John Mulligan [Tue, 26 Sep 2023 17:25:31 +0000 (13:25 -0400)]
cephadm: move a logging line closer to where the data is used

Move a logging line closer to where the data being logging is
used. This avoids having a dependency on logging in a fairly
simple function and should make moving the function in a future
commit easier.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agocephadm: move pathify & get_file_timestamp to file_utils
John Mulligan [Tue, 26 Sep 2023 16:56:35 +0000 (12:56 -0400)]
cephadm: move pathify & get_file_timestamp to file_utils

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agorgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH
Casey Bodley [Tue, 26 Sep 2023 16:16:52 +0000 (12:16 -0400)]
rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

ERR_SIGNATURE_NO_MATCH means that we found the given access key in
keystone, so we should use reject() instead of deny() to prevent
other engines like LocalEngine from looking up the access key again

this change causes us to return the SignatureDoesNotMatch error expected
by s3test case test_list_buckets_bad_auth()

Fixes: https://tracker.ceph.com/issues/62989
Signed-off-by: Casey Bodley <cbodley@redhat.com>