]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
11 months agovalgrind: UninitCondition under __run_exit_handlers suppression 58601/head
Mark Kogan [Mon, 21 Aug 2023 17:49:28 +0000 (17:49 +0000)]
valgrind: UninitCondition under __run_exit_handlers suppression

reqiered in CentOS / RHEL 9 & Ubuntu 22.04.1 LTS

Fixes: https://tracker.ceph.com/issues/62141
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 03616b4330080cadf20be440d92e33ca90de254a)

11 months agoqa/rgw: remove ragweed from all suites
Casey Bodley [Fri, 26 Jul 2024 15:57:49 +0000 (11:57 -0400)]
qa/rgw: remove ragweed from all suites

Signed-off-by: Casey Bodley <cbodley@redhat.com>
11 months agoqa: pin rgw/crypt suite temporarily to ubuntu_latest
Ali Maredia [Thu, 3 Aug 2023 16:45:01 +0000 (12:45 -0400)]
qa: pin rgw/crypt suite temporarily to ubuntu_latest

barbican tests are only passing on ubunutu latest,
this commit allows us to have a green crypt suite.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 863f8d789dbb02d70cbdf78790c63802cdd0f24d)

Conflicts:
ubuntu_latest.yaml: no qa/distros/all/ubuntu_latest.yaml, used
ubuntu_20.04.yaml

11 months agoqa: change admin_host to admin_url in barbican.py
Ali Maredia [Thu, 3 Aug 2023 16:39:12 +0000 (12:39 -0400)]
qa: change admin_host to admin_url in barbican.py

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 43069394c542f252e66453497cca8f195336d3f4)

11 months agoqa: bump keystone version in barbican.yaml
Ali Maredia [Mon, 17 Jul 2023 20:34:14 +0000 (16:34 -0400)]
qa: bump keystone version in barbican.yaml

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit fc12be0fcfabf3ae0040532ad06d91fcb7a94d55)

11 months agoqa: rgw/tempest suite only runs on ubuntu latest
Ali Maredia [Tue, 13 Jun 2023 17:21:14 +0000 (13:21 -0400)]
qa: rgw/tempest suite only runs on ubuntu latest

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 703effd45bf2c5d7cf8982e07e954a5f2cbb1902)

11 months agoqa: update rgw openstack versions
Tobias Urdin [Fri, 31 Mar 2023 07:10:20 +0000 (07:10 +0000)]
qa: update rgw openstack versions

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit fffa119424bad6e5c6b5ed436978d4319dd54a6b)

11 months agoqa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist
Casey Bodley [Mon, 21 Aug 2023 19:43:04 +0000 (15:43 -0400)]
qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Fixes: https://tracker.ceph.com/issues/62504
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4293d9bf117592a743ec70098cfb2bc1a9d38967)

Conflicts: git got confused about these; unchanged in original commit
qa/suites/rbd/device/ignore-pg-availability.yaml
qa/suites/upgrade/octopus-x/ignore-pg-availability.yaml

qa/suites/rgw/dbstore/ignore-pg-availability.yaml depended on
missing .qa symlink

11 months agorgw/notifications/test: install java for kafka build
Yuval Lifshitz [Sun, 15 Oct 2023 15:54:07 +0000 (15:54 +0000)]
rgw/notifications/test: install java for kafka build

also use kafka binaries insted of building from source

Fixes: https://tracker.ceph.com/issues/63205
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 2ab7c16737dbf97402907db74d7432a056f2fa97)

11 months agorgw/notifications/test: use dnf and systemctl to setup rabbitmq
Yuval Lifshitz [Sun, 15 Oct 2023 11:31:54 +0000 (11:31 +0000)]
rgw/notifications/test: use dnf and systemctl to setup rabbitmq

Fixes: https://tracker.ceph.com/issues/63205
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 1ffe4aca2a870600d2946e9432740848600a8ac4)

11 months agoqa/rgw/sts: keycloak task installs java manually
Casey Bodley [Fri, 22 Sep 2023 21:37:33 +0000 (17:37 -0400)]
qa/rgw/sts: keycloak task installs java manually

java had already been installed automatically before centos 9. add an
override to install the jdk-17 packages manually

Fixes: https://tracker.ceph.com/issues/62536
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 06f6a3e6e8ad65e75d640a128338ffa9dc246f4a)

Conflicts:
qa/suites/rgw/sts/tasks/1-keycloak.yaml moved from first.yaml

11 months agoqa/rgw/notifications: pin to centos_latest
Casey Bodley [Wed, 20 Sep 2023 15:11:37 +0000 (11:11 -0400)]
qa/rgw/notifications: pin to centos_latest

the subsuite had a supported-all-distro$/ subdirectory, but that only
contained centos_8.yaml. qa/tasks/rabbitmq.py is hardcoded to use 'yum'
and rpm packages, so replace supported-all-distro$ with a link to
centos_latest.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2d9348ad564f9199ae58062c1d12b7020cba0790)

11 months agoMerge pull request #58865 from zdover23/wip-doc-2024-07-26-backport-58838-to-quincy
Anthony D'Atri [Fri, 26 Jul 2024 00:36:36 +0000 (20:36 -0400)]
Merge pull request #58865 from zdover23/wip-doc-2024-07-26-backport-58838-to-quincy

quincy: doc/rbd/rbd-exclusive-locks: mention incompatibility with advisory locks

11 months agodoc/rbd/rbd-exclusive-locks: mention incompatibility with advisory locks 58865/head
Ilya Dryomov [Thu, 25 Jul 2024 07:48:10 +0000 (09:48 +0200)]
doc/rbd/rbd-exclusive-locks: mention incompatibility with advisory locks

This is due to both using the same name for the lock at the RADOS level
("rbd_lock").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 2e83b77c9f610f42765266ff4d4d7bc2a635eac8)

11 months agoMerge pull request #56231 from adk3798/quincy-docs-typo-replace-osd
Adam King [Thu, 25 Jul 2024 12:24:29 +0000 (08:24 -0400)]
Merge pull request #56231 from adk3798/quincy-docs-typo-replace-osd

quincy: doc: cephadm/services/osd: fix typo

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
11 months agoMerge pull request #58776 from zdover23/wip-doc-2024-07-24-backport-58524-to-quincy
Anthony D'Atri [Wed, 24 Jul 2024 22:28:58 +0000 (18:28 -0400)]
Merge pull request #58776 from zdover23/wip-doc-2024-07-24-backport-58524-to-quincy

quincy: doc/cephfs: streamline a paragraph

11 months agodoc/cephfs: streamline a paragraph 58776/head
Zac Dover [Wed, 10 Jul 2024 23:15:38 +0000 (09:15 +1000)]
doc/cephfs: streamline a paragraph

Streamline a paragraph in doc/cephfs/index.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 820f8ead8919483df37c751400bd6842d4c1bd92)

11 months agoMerge pull request #58731 from zdover23/wip-doc-2024-07-23-backport-58725-to-quincy
Anthony D'Atri [Mon, 22 Jul 2024 20:15:36 +0000 (16:15 -0400)]
Merge pull request #58731 from zdover23/wip-doc-2024-07-23-backport-58725-to-quincy

quincy: doc/dev_guide: add needs-upgrade-testing label info

11 months agoMerge pull request #58739 from zdover23/wip-doc-2024-07-23-backport-58600-to-quincy
Anthony D'Atri [Mon, 22 Jul 2024 20:12:17 +0000 (16:12 -0400)]
Merge pull request #58739 from zdover23/wip-doc-2024-07-23-backport-58600-to-quincy

quincy: doc/dev: add formatting to basic workflow

11 months agodoc/dev: add formatting to basic workflow 58739/head
Zac Dover [Mon, 15 Jul 2024 16:03:37 +0000 (02:03 +1000)]
doc/dev: add formatting to basic workflow

Add some formatting to a few sections near the end of
doc/dev/developer_guide_basic-workflow.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4c4063a09759dc3412d89ecafb9ac55a55e6bc01)

11 months agodoc/dev_guide: add needs-upgrade-testing label info 58731/head
Zac Dover [Mon, 22 Jul 2024 15:36:53 +0000 (01:36 +1000)]
doc/dev_guide: add needs-upgrade-testing label info

Add an explanation of the "needs-upgrade-testing" label and explain when
to use it.

Laura is the true author of this text.

Co-authored-by: Laura Flores <lflores@ibm.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 02d5a6a3371c9fa52a32124cf07da16b89a11e27)

11 months agoMerge pull request #58637 from zdover23/wip-doc-2024-07-16-backport-57685-to-quincy
Zac Dover [Mon, 22 Jul 2024 15:45:53 +0000 (01:45 +1000)]
Merge pull request #58637 from zdover23/wip-doc-2024-07-16-backport-57685-to-quincy

quincy: cephadm: use importlib.metadata for querying ceph_iscsi's version

Reviewed-by: Adam King <adking@redhat.com>
11 months agoMerge pull request #58698 from zdover23/wip-doc-2024-07-21-backport-58695-to-quincy
Anthony D'Atri [Sat, 20 Jul 2024 19:13:21 +0000 (15:13 -0400)]
Merge pull request #58698 from zdover23/wip-doc-2024-07-21-backport-58695-to-quincy

quincy: doc: Amend dev mailing list subscribe instructions.

12 months agodoc: Amend dev mailing list subscribe instructions. 58698/head
Paulo E. Castro [Sat, 20 Jul 2024 16:32:17 +0000 (17:32 +0100)]
doc: Amend dev mailing list subscribe instructions.

Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 3b0360bc2ea1e0b5f7a7c6b04ade483f77e4b5c6)

12 months agoMerge pull request #58682 from zdover23/wip-doc-2024-07-20-backport-58678-to-quincy
Anthony D'Atri [Fri, 19 Jul 2024 17:50:19 +0000 (13:50 -0400)]
Merge pull request #58682 from zdover23/wip-doc-2024-07-20-backport-58678-to-quincy

quincy: doc/cephadm/upgrade: ceph-ci containers are hosted by quay.ceph.io

12 months agoMerge pull request #58685 from zdover23/wip-doc-2024-07-20-backport-58671-to-quincy
Anthony D'Atri [Fri, 19 Jul 2024 17:44:31 +0000 (13:44 -0400)]
Merge pull request #58685 from zdover23/wip-doc-2024-07-20-backport-58671-to-quincy

quincy: doc/start: update mailing list links

12 months agodoc/start: update mailing list links 58685/head
Zac Dover [Fri, 19 Jul 2024 06:44:44 +0000 (16:44 +1000)]
doc/start: update mailing list links

Update the User and Dev mailing list subscription links. This commit is
made in response to a request from Dan van der Ster.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a242fe3d940d5764cd766dba8e42449501986765)

12 months agodoc/cephadm/upgrade: ceph-ci containers are hosted by quay.ceph.io 58682/head
Casey Bodley [Fri, 19 Jul 2024 14:10:38 +0000 (10:10 -0400)]
doc/cephadm/upgrade: ceph-ci containers are hosted by quay.ceph.io

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9c77963300fee804f249d512c980cdeac06f14f7)

12 months agoMerge pull request #58520 from ljflores/wip-add-c9-rados-quincy
Laura Flores [Thu, 18 Jul 2024 18:20:37 +0000 (13:20 -0500)]
Merge pull request #58520 from ljflores/wip-add-c9-rados-quincy

quincy: qa/distros: replace centos 8 references with centos 9 in the rados suite

12 months agoMerge pull request #58616 from zdover23/wip-doc-2024-07-16-backport-58596-to-quincy
Zac Dover [Thu, 18 Jul 2024 00:39:50 +0000 (10:39 +1000)]
Merge pull request #58616 from zdover23/wip-doc-2024-07-16-backport-58596-to-quincy

quincy: doc: clarify superuser note for ceph-fuse

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
12 months agoMerge pull request #57622 from sajibreadd/wip-65936-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:10:34 +0000 (16:10 -0700)]
Merge pull request #57622 from sajibreadd/wip-65936-quincy

quincy: os/bluestore: set rocksdb iterator bounds for Bluestore::_collection_list()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
12 months agoMerge pull request #57568 from amathuria/wip-66000-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:09:57 +0000 (16:09 -0700)]
Merge pull request #57568 from amathuria/wip-66000-quincy

quincy: mgr/pg_autoscaler: add check for norecover flag

Reviewed-by: Laura Flores <lflores@redhat.com>
12 months agoMerge pull request #57620 from sajibreadd/wip-66144-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:09:09 +0000 (16:09 -0700)]
Merge pull request #57620 from sajibreadd/wip-66144-quincy

quincy: osd: CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE flag is passed from ECBackend

Reviewed-by: Samuel Just <sjust@redhat.com>
12 months agoMerge pull request #57521 from pdvian/wip-65942-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:07:52 +0000 (16:07 -0700)]
Merge pull request #57521 from pdvian/wip-65942-quincy

quincy: mon/LogMonitor: Use generic cluster log level config

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
12 months agoMerge pull request #57243 from sajibreadd/wip-65358-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:06:57 +0000 (16:06 -0700)]
Merge pull request #57243 from sajibreadd/wip-65358-quincy

quincy: os/bluestore: expand BlueFS log if available space is insufficient

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
12 months agoMerge pull request #57198 from k0ste/wip-63316-quincy
Yuri Weinstein [Wed, 17 Jul 2024 23:05:59 +0000 (16:05 -0700)]
Merge pull request #57198 from k0ste/wip-63316-quincy

quincy: os/bluestore: fix crash caused by dividing by 0

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
12 months agocephadm: use importlib.metadata for querying ceph_iscsi's version 58637/head
Kefu Chai [Thu, 23 May 2024 23:21:51 +0000 (07:21 +0800)]
cephadm: use importlib.metadata for querying ceph_iscsi's version

use importlib.metadata for querying ceph_iscsi's version and fallback to
pkg_resources. as the former is only available in Python 3.8, while
the latter is deprecated.

Refs https://tracker.ceph.com/issues/66201
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit cbd4bede7f258823462f27cb15d86e13eb52c359)

Conflicts:
src/cephadm/cephadmlib/daemons/iscsi.py [ this file doesn't
  exist in quincy, get_version() lives in src/cephadm/cephadm ]

12 months agocephadm: extract python() helper to execute python statement
Kefu Chai [Thu, 23 May 2024 23:16:14 +0000 (07:16 +0800)]
cephadm: extract python() helper to execute python statement

to prepare for a change to use importlib, then fallback to
pkg_resources. as the former is only available in Python 3.8, while
the latter is deprecated.

Refs https://tracker.ceph.com/issues/66201
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit be1a8c2c43e97f91bd26554ba9dadc71c499865d)

Conflicts:
src/cephadm/cephadmlib/daemons/iscsi.py [ this file doesn't
  exist in quincy, get_version() lives in src/cephadm/cephadm ]

12 months agoqa/distros/podman: add centos 9 distro to podman 58520/head
Laura Flores [Tue, 16 Jul 2024 17:04:09 +0000 (12:04 -0500)]
qa/distros/podman: add centos 9 distro to podman

This commit, although a direct merge to quincy, is based
on c8873c6591d368e12907669c274fd3d6391e3f68.

The motivation is that centos 8 went end of life, so now
we need to use centos 9.

Signed-off-by: Laura Flores <lflores@ibm.com>
12 months agoqa/suites/rados/thrash-old-clients/0-distros$: test on ubuntu_20.04 and drop nautilus
Laura Flores [Tue, 16 Jul 2024 16:47:35 +0000 (11:47 -0500)]
qa/suites/rados/thrash-old-clients/0-distros$: test on ubuntu_20.04 and drop nautilus

Centos 8 has gone end of life, so we need to choose a different distro on which
to test thrash-old-clients.

thrash-old-clients tests should only support N-3 releases. Nautilus fits with
this, but unfortunately there is no overlapping distro between nautilus, pacific,
octopus, AND quincy (bionic was dropped from quincy, and nautilus does not build
focal). As such, we are only able to test N-2.

Proof that focal is not available for octopus (this is where the test would search for packages):
https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nautilus

Signed-off-by: Laura Flores <lflores@ibm.com>
12 months agoqa/suites/orch/cephadm/workunits/task/test_iscsi_container: switch to centos 9
Laura Flores [Wed, 10 Jul 2024 20:32:53 +0000 (15:32 -0500)]
qa/suites/orch/cephadm/workunits/task/test_iscsi_container: switch to centos 9

Centos 8 went end of life (see https://docs.ceph.com/en/latest/start/os-recommendations/#platforms),
so we will now test on centos 9 for quincy.

Signed-off-by: Laura Flores <lflores@ibm.com>
12 months agodoc: elide superuser discussion from mount prereq 58616/head
Patrick Donnelly [Mon, 15 Jul 2024 14:18:15 +0000 (10:18 -0400)]
doc: elide superuser discussion from mount prereq

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7290bdce6f4dd93be88be6ad54030fa90a8c1ec0)

12 months agodoc: clarify superuser note for ceph-fuse
Patrick Donnelly [Mon, 15 Jul 2024 14:12:24 +0000 (10:12 -0400)]
doc: clarify superuser note for ceph-fuse

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d100de9c9ce884beee6e9551b2b221c686b775b5)

12 months agoMerge pull request #58577 from zdover23/wip-doc-2024-07-14-backport-58451-to-quincy
Zac Dover [Mon, 15 Jul 2024 13:21:20 +0000 (23:21 +1000)]
Merge pull request #58577 from zdover23/wip-doc-2024-07-14-backport-58451-to-quincy

quincy: doc/dev: edit "Principles for format change"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
12 months agoMerge pull request #58580 from zdover23/wip-doc-2024-07-15-backport-58574-to-quincy
Anthony D'Atri [Sun, 14 Jul 2024 17:23:08 +0000 (13:23 -0400)]
Merge pull request #58580 from zdover23/wip-doc-2024-07-15-backport-58574-to-quincy

quincy: doc/cephfs: edit "is mount helper present"

12 months agodoc/cephfs: edit "is mount helper present" 58580/head
Zac Dover [Sun, 14 Jul 2024 06:40:14 +0000 (16:40 +1000)]
doc/cephfs: edit "is mount helper present"

Edit the section called "Is mount helper present?", the title of which
prior to this commit was "Is mount helper is present?". Other small
disambiguating improvements have been made to the text in the section.
An unselectable prompt has been added before a command.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0ebfb3eda8e2fc88662530bbddc84f2efcc4e13c)

12 months agodoc/dev: edit "Principles for format change" 58577/head
Zac Dover [Sun, 7 Jul 2024 03:59:15 +0000 (13:59 +1000)]
doc/dev: edit "Principles for format change"

Improve "Principles for format change" in doc/dev/encoding.rst. This
commit started as a response to Anthony D'Atri's suggestion here: https://github.com/ceph/ceph/pull/58299/files#r1656985564

Review of this section suggested to me that certain minor English usage
improvements would be of benefit. The numbered lists in this section
could still be made a bit clearer.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 570797e5588b67b8c72e5297b61f84d9aa48dc45)

12 months agoMerge pull request #58342 from idryomov/wip-66570-quincy
Ilya Dryomov [Sat, 13 Jul 2024 09:21:45 +0000 (11:21 +0200)]
Merge pull request #58342 from idryomov/wip-66570-quincy

quincy: librbd: make diff-iterate in fast-diff mode aware of encryption

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #58539 from idryomov/wip-66886-quincy
Ilya Dryomov [Sat, 13 Jul 2024 08:40:24 +0000 (10:40 +0200)]
Merge pull request #58539 from idryomov/wip-66886-quincy

quincy: qa: account for rbd_trash object in krbd_data_pool.sh + related ceph{,adm} task fixes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #56414 from zdover23/wip-doc-2024-03-23-backport-56379-to-quincy
Ilya Dryomov [Sat, 13 Jul 2024 07:20:31 +0000 (09:20 +0200)]
Merge pull request #56414 from zdover23/wip-doc-2024-03-23-backport-56379-to-quincy

quincy: doc/install: fix typos in openEuler-installation doc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
12 months agodoc/install: fix typos in openEuler-installation doc 56414/head
Rongqi Sun [Fri, 22 Mar 2024 03:12:09 +0000 (03:12 +0000)]
doc/install: fix typos in openEuler-installation doc

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
(cherry picked from commit 593ab1ad41467898c57c44baeb947bd238cba645)

12 months agolibrbd: make diff-iterate in fast-diff mode aware of encryption 58342/head
Ilya Dryomov [Thu, 20 Jun 2024 19:13:56 +0000 (21:13 +0200)]
librbd: make diff-iterate in fast-diff mode aware of encryption

diff-iterate wasn't updated when librbd was being prepared to support
encryption in commit 8d6a47933269 ("librbd: add crypto image dispatch
layer").  This is even noted in [1]:

> The two places I skipped for now are DiffIterate and TrimRequest.

CryptoImageDispatch has since been removed, but diff-iterate in
fast-diff mode is still unaware of encryption and just assumes that all
offsets are raw.  This means that the callback gets invoked with
incorrect image offsets when encryption is loaded.  For example, for
a LUKS1-formatted image with some data at offsets 0 and 20971520,
diff-iterate with encryption loaded reports

  0~4194304
  4194304~4194304
  25165824~4194304

instead of

  0~4194304
  20971520~4194304

as "exists".  For any piece of code that is using diff-iterate to
optimize block-by-block processing (e.g. copy an encrypted source image
to a differently-encrypted destination image), this is fatal: it would
skip processing block 20971520 which has data and instead process block
25165824 which doesn't have any data and was to be skipped, producing
a corrupted destination image.

[1] https://github.com/ceph/ceph/pull/37935#issue-735278403

Fixes: https://tracker.ceph.com/issues/66570
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit cdeb0efce3f9f857ad6d5b7ff3965f3292cb571a)

Conflicts:
src/librbd/api/DiffIterate.cc [ ImageArea support not in
  quincy ]
src/test/librbd/test_librbd.cc [ commit 4a5a0a5dd82b ("librbd:
  add cloned images encryption API") not in quincy

12 months agotest/librbd: extend DiffIterateTest.DiffIterateDeterministic{,PP}
Ilya Dryomov [Fri, 21 Jun 2024 13:56:27 +0000 (15:56 +0200)]
test/librbd: extend DiffIterateTest.DiffIterateDeterministic{,PP}

Currently we are laying data only at the beginning of an object.
Extend the skeletons to write to three different offsets in the middle
and also at the end of the object.

Separately, make C and C++ API test variants slightly different in
terms of offsets being targeted to not go through exactly the same
scenario twice.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit da69d1fdbf69cf37ba8b75ea9a230634bb25ae76)

12 months agotest/librbd: refactor DiffIterateTest.DiffIterateDeterministic{,PP}
Ilya Dryomov [Fri, 21 Jun 2024 13:53:05 +0000 (15:53 +0200)]
test/librbd: refactor DiffIterateTest.DiffIterateDeterministic{,PP}

In preparation for extending coverage, extract test logic into a couple
of parametrized helpers.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1df87998dc30be6e3b4c90f5833539c7ccc70911)

12 months agoMerge pull request #58210 from idryomov/wip-66418-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:06:54 +0000 (13:06 -0700)]
Merge pull request #58210 from idryomov/wip-66418-quincy

quincy: librbd: diff-iterate shouldn't crash on an empty byte range

Reviewed-by: Mykola Golub <mgolub@suse.com>
12 months agoMerge pull request #57764 from idryomov/wip-66232-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:06:13 +0000 (13:06 -0700)]
Merge pull request #57764 from idryomov/wip-66232-quincy

quincy: qa/suites/rbd: override extra_system_packages directly on install task

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #57604 from ajarr/wip-66115-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:05:29 +0000 (13:05 -0700)]
Merge pull request #57604 from ajarr/wip-66115-quincy

quincy: librbd: create rbd_trash object during pool initialization and namespace creation

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
12 months agoMerge pull request #57569 from idryomov/wip-65813-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:04:51 +0000 (13:04 -0700)]
Merge pull request #57569 from idryomov/wip-65813-quincy

quincy: librbd: don't crash on a zero-length read if buffer is NULL

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #57308 from idryomov/wip-65624-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:04:20 +0000 (13:04 -0700)]
Merge pull request #57308 from idryomov/wip-65624-quincy

quincy: pybind/rbd: expose CLONE_FORMAT and FLATTEN image options

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #57305 from idryomov/wip-65487-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:03:48 +0000 (13:03 -0700)]
Merge pull request #57305 from idryomov/wip-65487-quincy

quincy: rbd-mirror: clean up stale pool replayers and callouts better

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #57090 from idryomov/wip-65573-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:03:17 +0000 (13:03 -0700)]
Merge pull request #57090 from idryomov/wip-65573-quincy

quincy: librbd: make group and group snapshot IDs more random

Reviewed-by: Mykola Golub <mgolub@suse.com>
12 months agoMerge pull request #56366 from Svelar/wip-65031-quincy
Yuri Weinstein [Fri, 12 Jul 2024 20:02:22 +0000 (13:02 -0700)]
Merge pull request #56366 from Svelar/wip-65031-quincy

quincy: ceph.spec.in: add support for openEuler OS

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
12 months agoMerge pull request #58523 from cbodley/wip-66898-quincy
Casey Bodley [Fri, 12 Jul 2024 19:20:16 +0000 (20:20 +0100)]
Merge pull request #58523 from cbodley/wip-66898-quincy

quincy: install-deps.sh, do_cmake.sh: almalinux is another el flavour

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
Reviewed-by: Dan van der Ster <dan.vanderster@clyso.com>
12 months agoMerge pull request #58548 from idryomov/wip-test-cookie-quincy
Ilya Dryomov [Fri, 12 Jul 2024 16:16:12 +0000 (18:16 +0200)]
Merge pull request #58548 from idryomov/wip-test-cookie-quincy

quincy: test/librbd: clean up unused TEST_COOKIE variable

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agotest/ librbd: clean up unused 'TEST_COOKIE' 58548/head
Rongqi Sun [Tue, 9 Jul 2024 02:55:27 +0000 (02:55 +0000)]
test/ librbd: clean up unused 'TEST_COOKIE'

Clean them up because
```
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/librbd/exclusive_lock/test_mock_PreAcquireRequest.cc:47:26: error: unused variable 'TEST_COOKIE' [-Werror,-Wunused-const-variable]
static const std::string TEST_COOKIE("auto 123");
                         ^
1 error generated.
```

Ref: https://jenkins.ceph.com/job/ceph-pull-requests-arm64/59242/consoleFull#-5120798632a811ea2-3e7b-466b-84b4-d13df7e35809

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
(cherry picked from commit 49869222c816023f696ff2a42898e2d9f3f54215)

12 months agoqa/tasks/cephadm: drop --cluster and sudo in create_rbd_pool() 58539/head
Ilya Dryomov [Wed, 10 Jul 2024 09:53:05 +0000 (11:53 +0200)]
qa/tasks/cephadm: drop --cluster and sudo in create_rbd_pool()

The cluster (name) is already specified in the arguments passed to
_shell() and this command doesn't need privileges.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 26cee25873d5c91c704326fd0b708c53b353dd11)

12 months agoqa/tasks/cephadm: don't wait for OSDs in create_rbd_pool()
Ilya Dryomov [Wed, 10 Jul 2024 09:02:47 +0000 (11:02 +0200)]
qa/tasks/cephadm: don't wait for OSDs in create_rbd_pool()

This fails because teuthology.wait_until_osds_up() wants to use
adjust-ulimits wrapper which isn't available in "cephadm shell"
environment.  The whole thing is also redundant because cephadm task
is supposed to wait for OSDs to come up earlier, in ceph_osds().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0bda782952a81820752acdbf0b3ab9dbed56fb64)

12 months agoqa/tasks: initialize rbd pool instead of just tagging it
Ilya Dryomov [Wed, 10 Jul 2024 07:10:02 +0000 (09:10 +0200)]
qa/tasks: initialize rbd pool instead of just tagging it

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fcc440da9aaa6585872e830a5e39aefce1df82fe)

12 months agoqa: krbd_data_pool.sh: account for rbd_trash object
Ilya Dryomov [Wed, 10 Jul 2024 07:10:02 +0000 (09:10 +0200)]
qa: krbd_data_pool.sh: account for rbd_trash object

... created since commit 0bcb5cfce9a2 ("librbd/image: create rbd_trash
object during RBD pool initialization").

Fixes: https://tracker.ceph.com/issues/66886
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f8e5aecfc75bf7d32523e7a8d2d6034b434d216a)

12 months agoinstall-deps.sh, do_cmake.sh: almalinux is another el flavour 58523/head
Dan van der Ster [Thu, 5 Oct 2023 16:50:36 +0000 (09:50 -0700)]
install-deps.sh, do_cmake.sh: almalinux is another el flavour

Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
(cherry picked from commit 6e85f76385c3cffd860a8a3054bf14ce85a43b59)

Conflicts:
do_cmake.sh add rocky too
install-deps.sh add rocky too

12 months agoqa/distros/container-hosts: add centos 9 to container hosts
Laura Flores [Wed, 10 Jul 2024 19:49:25 +0000 (14:49 -0500)]
qa/distros/container-hosts: add centos 9 to container hosts

This is a direct merge to quincy that is based on the following
commit: https://github.com/ceph/ceph/commit/c8873c6591d368e12907669c274fd3d6391e3f68
It is not directly backported due to backport complexities.

Signed-off-by: Laura Flores <lflores@ibm.com>
12 months agoqa/distros: replace centos8 and rhel8 with centos9
Laura Flores [Wed, 10 Jul 2024 19:36:31 +0000 (14:36 -0500)]
qa/distros: replace centos8 and rhel8 with centos9

This commit is based on https://github.com/ceph/ceph/commit/7a1dce1ebd883741b5003b9e18d4765526cbbb3e,
but due to backport complexities, it is a direct merge to quincy.

Centos 8 went end of life (as did rhel 8), so we will now
test with centos 9 for quincy.
Ref: https://docs.ceph.com/en/latest/start/os-recommendations/#platforms

Signed-off-by: Laura Flores <lflores@ibm.com>
12 months agoMerge pull request #58490 from zdover23/wip-doc-2024-07-10-backport-58484-to-quincy
Zac Dover [Wed, 10 Jul 2024 00:27:10 +0000 (10:27 +1000)]
Merge pull request #58490 from zdover23/wip-doc-2024-07-10-backport-58484-to-quincy

quincy: doc: Correct link to "Device management"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
12 months agodoc: Correct link to "Device management" 58490/head
Matthew Vernon [Tue, 9 Jul 2024 16:02:31 +0000 (17:02 +0100)]
doc: Correct link to "Device management"

The current link goes to `cephadm/services/rados/operations/devices` which is a 404, it instead needs to be `rados/operations/devices/`.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit d1a7cc4978adacd4e11df2d3e0886b191b118cc1)

12 months agoMerge pull request #58445 from ceph/wip-yuriw-quincy-client-x
Yuri Weinstein [Tue, 9 Jul 2024 13:17:54 +0000 (06:17 -0700)]
Merge pull request #58445 from ceph/wip-yuriw-quincy-client-x

qa/tests: added client-upgrade-quincy-squid tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
12 months agoqa/tests: added client-upgrade-quincy-squid tests 58445/head
Yuri Weinstein [Fri, 5 Jul 2024 17:33:23 +0000 (10:33 -0700)]
qa/tests: added client-upgrade-quincy-squid tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
12 months agoMerge pull request #58433 from zdover23/wip-doc-2024-07-05-backport-58368-to-quincy
Anthony D'Atri [Thu, 4 Jul 2024 20:31:41 +0000 (16:31 -0400)]
Merge pull request #58433 from zdover23/wip-doc-2024-07-05-backport-58368-to-quincy

quincy: doc/rados: document manually passing search domain

12 months agoMerge pull request #58309 from zdover23/wip-doc-2024-06-27-backport-58287-to-quincy
Zac Dover [Thu, 4 Jul 2024 18:29:59 +0000 (04:29 +1000)]
Merge pull request #58309 from zdover23/wip-doc-2024-06-27-backport-58287-to-quincy

quincy: doc/rados: update how to install c++ header files

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
12 months agodoc/rados: document manually passing search domain 58433/head
Zac Dover [Mon, 1 Jul 2024 06:36:02 +0000 (16:36 +1000)]
doc/rados: document manually passing search domain

Document how to manually pass the search domain to "mon_dns_srv_name" in
doc/rados/configuration/mon-lookup-dns.rst.

This commit is made in response to a request by Lander Duncan that was made on the [ceph-users] mailing list, and can be seen here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/F7V4CWLIYCAJ4JXI2JLNY6QPCFPR4SLA/

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 98938a0312dd0c8e0b293ed9aa2e0760cc9619fa)

12 months agoMerge pull request #58391 from zdover23/wip-doc-2024-07-02-backport-58092-to-quincy
Zac Dover [Wed, 3 Jul 2024 10:17:08 +0000 (20:17 +1000)]
Merge pull request #58391 from zdover23/wip-doc-2024-07-02-backport-58092-to-quincy

quincy: doc/start: remove mention of Centos 8 support

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
12 months agodoc/start: remove mention of Centos 8 support 58391/head
Zac Dover [Mon, 17 Jun 2024 15:09:44 +0000 (01:09 +1000)]
doc/start: remove mention of Centos 8 support

Remove information so that it is clear that Centos 8 will no longer be
supported by Ceph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6b149e4f6ab2ac716e73f42b0d6c67482a768da9)

12 months agoMerge pull request #58372 from zdover23/wip-doc-2024-07-01-backport-58366-to-quincy
Anthony D'Atri [Mon, 1 Jul 2024 15:24:51 +0000 (11:24 -0400)]
Merge pull request #58372 from zdover23/wip-doc-2024-07-01-backport-58366-to-quincy

quincy: doc/cephfs: update cephfs-shell link

12 months agoMerge pull request #58365 from zdover23/wip-doc-2024-07-01-backport-55577-to-quincy
Anthony D'Atri [Mon, 1 Jul 2024 13:11:44 +0000 (09:11 -0400)]
Merge pull request #58365 from zdover23/wip-doc-2024-07-01-backport-55577-to-quincy

quincy: doc/start: fix wording & syntax

12 months agodoc/cephfs: update cephfs-shell link 58372/head
Zac Dover [Mon, 1 Jul 2024 05:57:40 +0000 (15:57 +1000)]
doc/cephfs: update cephfs-shell link

Repair the link to cephfs-shell.rst in doc/cephfs/cephfs-shell.rst that
was broken in https://github.com/ceph/ceph/pull/41165/ when
doc/cephfs/cephfs-shell.rst was moved to doc/man/8/cephfs-shell.rst.

This commit is made in response to a request by Lander Duncan that was
made on the [ceph-users] mailing list, and can be seen here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/F7V4CWLIYCAJ4JXI2JLNY6QPCFPR4SLA/

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4f8903cf80a94596a1b17b4a07affb874f21ae76)

12 months agodoc/start: fix wording & syntax 58365/head
Piotr Parczewski [Wed, 14 Feb 2024 10:15:09 +0000 (11:15 +0100)]
doc/start: fix wording & syntax

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit a8b27af84910d324023a37e616f42d6d3faa4ecf)

12 months agoMerge pull request #58361 from zdover23/wip-doc-2024-06-30-backport-58358-to-quincy
Anthony D'Atri [Sat, 29 Jun 2024 19:23:42 +0000 (15:23 -0400)]
Merge pull request #58361 from zdover23/wip-doc-2024-06-30-backport-58358-to-quincy

quincy: doc/cephfs: take Anthony's suggestion

12 months agodoc/cephfs: take Anthony's suggestion 58361/head
Zac Dover [Sat, 29 Jun 2024 12:11:55 +0000 (22:11 +1000)]
doc/cephfs: take Anthony's suggestion

Incorporate Anthony's suggestion in
https://github.com/ceph/ceph/pull/57567#discussion_r1607016050 into
doc/cephfs/fs-volumes.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7a78dda0a92339db27359f91f0cb081703e2e7f9)

12 months agoMerge pull request #58306 from zdover23/wip-doc-2024-06-27-backport-58299-to-quincy
Anthony D'Atri [Thu, 27 Jun 2024 14:17:55 +0000 (10:17 -0400)]
Merge pull request #58306 from zdover23/wip-doc-2024-06-27-backport-58299-to-quincy

quincy: doc/dev: Fix typos in encoding.rst

12 months agodoc/rados: update how to install c++ header files 58309/head
Pere Diaz Bou [Wed, 26 Jun 2024 13:57:47 +0000 (15:57 +0200)]
doc/rados: update how to install c++ header files

In this example librados2-devel only install C header files on fedora 40,
therefore I added libradospp-devel to the command to include C++ header files.

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
(cherry picked from commit 7e87441601db2379488fab62e900cd834086f26d)

12 months agodoc/dev: Fix typos in encoding.rst 58306/head
N Balachandran [Thu, 27 Jun 2024 07:22:09 +0000 (12:52 +0530)]
doc/dev: Fix typos in encoding.rst

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 9f43c67a639958a13fef198ffe62cf5241f1f53c)

12 months agoMerge pull request #58227 from vshankar/wip-quincy-override-client-features
Venky Shankar [Wed, 26 Jun 2024 13:00:26 +0000 (18:30 +0530)]
Merge pull request #58227 from vshankar/wip-quincy-override-client-features

quincy: override client features

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
12 months agoMerge pull request #58273 from zdover23/wip-doc-2024-06-26-backport-58226-to-quincy
Anthony D'Atri [Wed, 26 Jun 2024 12:33:17 +0000 (08:33 -0400)]
Merge pull request #58273 from zdover23/wip-doc-2024-06-26-backport-58226-to-quincy

quincy: doc/rados: edit troubleshooting-osd.rst

12 months agodoc/rados: edit troubleshooting-osd.rst 58273/head
Zac Dover [Mon, 24 Jun 2024 10:32:30 +0000 (20:32 +1000)]
doc/rados: edit troubleshooting-osd.rst

Make minor changes to the "Debugging Slow Requests" section of
doc/rados/troubleshooting/troubleshooting-osd.rst in preparation
for an expansion of this section in response to a reqeust from Joel
Davidow.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8b211b9c7f9b173bb12028eb7108d13867883d6e)

12 months agoMerge pull request #58259 from zdover23/wip-doc-2024-06-25-backport-58243-to-quincy
Zac Dover [Wed, 26 Jun 2024 05:52:42 +0000 (15:52 +1000)]
Merge pull request #58259 from zdover23/wip-doc-2024-06-25-backport-58243-to-quincy

quincy: doc/rados: credit Prashant for a procedure

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
12 months agodoc/rados: credit Prashant for a procedure 58259/head
Zac Dover [Tue, 25 Jun 2024 04:49:13 +0000 (14:49 +1000)]
doc/rados: credit Prashant for a procedure

Credit Prashant D for creating the stretch-mode workaround procedure for
retrieving the correct size of datacenters.

Follows: https://github.com/ceph/ceph/pull/58109

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2e777cb4f8bd682b807b97074f930826aa902db9)

12 months agoclient: allow overriding client features 58227/head
Patrick Donnelly [Thu, 2 May 2024 00:51:59 +0000 (20:51 -0400)]
client: allow overriding client features

For testing purposes.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d9239f9375c1ae92a4990950f40078766bd912e8)

 Conflicts:
src/client/Client.h

Resolve a minor conflict involving Client::cap_auths

12 months agoMerge pull request #58204 from zdover23/wip-doc-2024-06-23-backport-58200-to-quincy
Zac Dover [Mon, 24 Jun 2024 00:32:08 +0000 (10:32 +1000)]
Merge pull request #58204 from zdover23/wip-doc-2024-06-23-backport-58200-to-quincy

quincy: doc/start: add links to Beginner's Guide

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
12 months agotest/librbd: cover empty and invalid snap id ranges for diff-iterate 58210/head
Ilya Dryomov [Tue, 11 Jun 2024 17:02:23 +0000 (19:02 +0200)]
test/librbd: cover empty and invalid snap id ranges for diff-iterate

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4f8aca21960088e80c4917ac2bb8df49ea16c28a)

12 months agolibrbd: diff-iterate shouldn't crash on an empty byte range
Ilya Dryomov [Tue, 11 Jun 2024 16:10:47 +0000 (18:10 +0200)]
librbd: diff-iterate shouldn't crash on an empty byte range

Commit 0b5ba5fedf70 ("librbd/object_map: add support for ranged
diff-iterate") introduced a regression for the case when whole_object
parameter is set to true.  Despite DiffRequest being called into and
another DiffIterate potentially being spawned recursively, an empty
byte range previously happened to make it.

Bail on an empty byte range early just like we have always done on an
empty snap id range (i.e. when start and end versions are the same).

Fixes: https://tracker.ceph.com/issues/66418
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a3441a701963fef155fdda7d8f949d73bae26b69)

12 months agolibrbd: assert on m_length in calc_object_diff_range()
Ilya Dryomov [Tue, 11 Jun 2024 14:03:22 +0000 (16:03 +0200)]
librbd: assert on m_length in calc_object_diff_range()

A non-empty byte range is required by both last_period_off calculation
and Striper::file_to_extents().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6b5f0088d799a72a840fae685b44b0f31eee6be8)