]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoqa/s3tests: run with tox instead of bootstrap+nose 49826/head
Casey Bodley [Sun, 22 Jan 2023 16:37:56 +0000 (11:37 -0500)]
qa/s3tests: run with tox instead of bootstrap+nose

Fixes: https://tracker.ceph.com/issues/58512
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoqa/s3tests: add tox task before every s3tests task
Casey Bodley [Sun, 22 Jan 2023 19:01:15 +0000 (14:01 -0500)]
qa/s3tests: add tox task before every s3tests task

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49817 from ivancich/wip-flight-copyright-license-info
J. Eric Ivancich [Sat, 21 Jan 2023 01:32:17 +0000 (20:32 -0500)]
Merge pull request #49817 from ivancich/wip-flight-copyright-license-info

rgw: add copyright & licensing to flight integration source files

Reviewed-By: Adam Emerson <aemerson@redhat.com>
2 years agorgw: add copyright & licensing to flight integration source files 49817/head
J. Eric Ivancich [Fri, 20 Jan 2023 20:55:12 +0000 (15:55 -0500)]
rgw: add copyright & licensing to flight integration source files

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agoMerge pull request #49791 from idryomov/wip-doc-exclusive-lock-transitions
Ilya Dryomov [Fri, 20 Jan 2023 13:08:10 +0000 (14:08 +0100)]
Merge pull request #49791 from idryomov/wip-doc-exclusive-lock-transitions

doc/rbd/rbd-exclusive-locks: warn about automatic lock transitions

Reviewed-by: Ramana Raja <rraja@redhat.com>
2 years agoMerge pull request #49784 from SUSE/wip-fix-58501
Ilya Dryomov [Fri, 20 Jan 2023 13:05:22 +0000 (14:05 +0100)]
Merge pull request #49784 from SUSE/wip-fix-58501

ceph.spec.in: Replace %usrmerged macro with regular version check

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #49764 from zdover23/wip-doc-2023-01-17-ceph-volume-lvm-encryption...
zdover23 [Fri, 20 Jan 2023 10:50:42 +0000 (20:50 +1000)]
Merge pull request #49764 from zdover23/wip-doc-2023-01-17-ceph-volume-lvm-encryption-refine

doc/ceph-volume: refine encryption.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rbd/rbd-exclusive-locks: warn about automatic lock transitions 49791/head
Ilya Dryomov [Thu, 19 Jan 2023 12:21:40 +0000 (13:21 +0100)]
doc/rbd/rbd-exclusive-locks: warn about automatic lock transitions

A lot of people aren't aware of automatic lock transitions and
wrongfully assume that exclusive lock means that the image remains
locked for as long as the client is running.  Redo the explanation
and add a warning.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agodoc/rbd/rbd-exclusive-locks: don't mention "profile rbd" requirement twice
Ilya Dryomov [Wed, 18 Jan 2023 19:20:19 +0000 (20:20 +0100)]
doc/rbd/rbd-exclusive-locks: don't mention "profile rbd" requirement twice

It's (much better) described in the Blocklisting section.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #49798 from adamemerson/wip-radosgw-cr-test-build-fix
Casey Bodley [Fri, 20 Jan 2023 01:21:30 +0000 (20:21 -0500)]
Merge pull request #49798 from adamemerson/wip-radosgw-cr-test-build-fix

build: Fix build of `radosgw-cr-test`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agobuild: Fix build of `radosgw-cr-test` 49798/head
Adam C. Emerson [Thu, 19 Jan 2023 21:51:58 +0000 (16:51 -0500)]
build: Fix build of `radosgw-cr-test`

Remove dependency on `libneorados_cls_fifo` removed in commit
`bf222906b01cbb9e8d55ae4685004a22cddcc138`.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agoMerge pull request #48898 from ceph/wip-rgw-multisite-reshard-reef
Adam C. Emerson [Thu, 19 Jan 2023 17:26:52 +0000 (12:26 -0500)]
Merge pull request #48898 from ceph/wip-rgw-multisite-reshard-reef

rgw: multisite stabilization for reef

Reviewed-By: Adam Emerson <aemerson@redhat.com>
2 years agoMerge pull request #48848 from stefan-chivu/rbd-wnbd-live-resize
Ilya Dryomov [Thu, 19 Jan 2023 16:29:36 +0000 (17:29 +0100)]
Merge pull request #48848 from stefan-chivu/rbd-wnbd-live-resize

rbd: live resize support for rbd-wnbd

Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agodoc/ceph-volume: refine encryption.rst 49764/head
Zac Dover [Tue, 17 Jan 2023 09:45:23 +0000 (19:45 +1000)]
doc/ceph-volume: refine encryption.rst

Improve the word choice and grammar of
doc/ceph-volume/lvm/encryption.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49280 from stefan-chivu/dokan-file-dir-access
Venky Shankar [Thu, 19 Jan 2023 13:44:41 +0000 (19:14 +0530)]
Merge pull request #49280 from stefan-chivu/dokan-file-dir-access

dokan: Made file/dir access mode configurable

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #49759 from rhcs-dashboard/fix-pg-metrics
Avan [Thu, 19 Jan 2023 09:34:41 +0000 (15:04 +0530)]
Merge pull request #49759 from rhcs-dashboard/fix-pg-metrics

mgr/prometheus: export zero valued pg state metrics

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoceph.spec.in: Replace %usrmerged macro with regular version check 49784/head
Tim Serong [Thu, 19 Jan 2023 07:26:52 +0000 (18:26 +1100)]
ceph.spec.in: Replace %usrmerged macro with regular version check

%usrmerged is deprecated in favour of %if 0%{?suse_version} < 1550
(see https://en.opensuse.org/openSUSE:Usr_merge for details)

Fixes: https://tracker.ceph.com/issues/58501
Signed-off-by: Tim Serong <tserong@suse.com>
2 years agoMerge pull request #49780 from zdover23/wip-doc-2023-01-19-cephadm-install-link-fix
colemitchell [Thu, 19 Jan 2023 02:36:59 +0000 (21:36 -0500)]
Merge pull request #49780 from zdover23/wip-doc-2023-01-19-cephadm-install-link-fix

doc/install: link to "cephadm installing ceph"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/install: link to "cephadm installing ceph" 49780/head
Zac Dover [Thu, 19 Jan 2023 01:50:17 +0000 (11:50 +1000)]
doc/install: link to "cephadm installing ceph"

Link to "Installing Ceph" in the cephadm documentation instead of (as
was the case before this commit) to the cephadm overview page. Anyone
who clicks on the "cephadm" link in the context of the
doc/install/index.rst page is more likely to expect installation
instructions than to expect an explanation of what cephadm is.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49734 from linuxkidd/wip-rgw-gap-orphan-tool-error-checks
J. Eric Ivancich [Wed, 18 Jan 2023 20:30:21 +0000 (15:30 -0500)]
Merge pull request #49734 from linuxkidd/wip-rgw-gap-orphan-tool-error-checks

rgw/rgw-orphan-list: refactor and add more checks to the tool

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agorgw/rgw-orphan-list: refactor / add error checking 49734/head
Michael J. Kidd [Wed, 11 Jan 2023 19:53:50 +0000 (14:53 -0500)]
rgw/rgw-orphan-list: refactor / add error checking

standardize log output
add check that supplied pool[s] exist on the cluster
add return code checks for all commands that can affect output
add check for empty intermediate files
add usage output, including a -h command line option
add command line option to specify temporary directory to use
  ( instead of /tmp )
require command line supplied pools to be encased in double quotes "
  when multiple pools are provided
add warning for indexless buckets
move major segments to separate functions

Signed-off-by: Michael J. Kidd <linuxkidd@gmail.com>
2 years agoMerge pull request #48632 from adamemerson/wip-57562
Casey Bodley [Wed, 18 Jan 2023 15:41:31 +0000 (10:41 -0500)]
Merge pull request #48632 from adamemerson/wip-57562

rgw: Fix race condition on FIFO client on new head creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Krunal Chheda <kchheda3@bloomberg.net>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoqa: add windows run-tests.ps1 48848/head
Lucian Petrut [Fri, 6 Jan 2023 12:29:02 +0000 (14:29 +0200)]
qa: add windows run-tests.ps1

We have a few Python rbd-wnbd tests that are invoked explicitly
by the ceph-build scripts [1].

There are a few issues with that:

* it's a separate repo that has to be updated whenever we add new
  tests
* new tests that reside in the ceph repo will not be executed by
  the PR check
* some tests may be missing in case of older branches

For this reason, we're adding a new script as part of the Ceph
repo that will take care of invoking the Windows rbd-wnbd tests.

The ceph-build script has already been updated accordingly [2].

[1] https://github.com/ceph/ceph-build/blob/main/scripts/ceph-windows/run_tests#L73-L80
[2] https://github.com/ceph/ceph-build/pull/2094

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-Authored-By: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2 years agoMerge pull request #47794 from rhcs-dashboard/auth-add
Pere Diaz Bou [Wed, 18 Jan 2023 09:06:29 +0000 (10:06 +0100)]
Merge pull request #47794 from rhcs-dashboard/auth-add

mgr/dashboard: create authx users

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #49765 from rhcs-dashboard/fix-bucket-encryption-checkbox
Aashish Sharma [Wed, 18 Jan 2023 04:59:31 +0000 (10:29 +0530)]
Merge pull request #49765 from rhcs-dashboard/fix-bucket-encryption-checkbox

mgr/dashboard: fix bucket encryption checkbox

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #48729 from ivancich/wip-flight-select-add-flight
J. Eric Ivancich [Tue, 17 Jan 2023 23:22:54 +0000 (18:22 -0500)]
Merge pull request #48729 from ivancich/wip-flight-select-add-flight

rgw: initial commit adding Arrow Flight functionality

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #48509 from jliebl-git/jliebl-ldap-fix
J. Eric Ivancich [Tue, 17 Jan 2023 22:57:08 +0000 (17:57 -0500)]
Merge pull request #48509 from jliebl-git/jliebl-ldap-fix

rgw: LDAP fix resource leak with wrong credentials

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #49636 from liangmingyuanneo/wip-read-olh-log
J. Eric Ivancich [Tue, 17 Jan 2023 22:54:54 +0000 (17:54 -0500)]
Merge pull request #49636 from liangmingyuanneo/wip-read-olh-log

rgw: remove guard_reshard in bucket_index_read_olh_log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49463 from ivancich/wip-log-follow-bi-list
J. Eric Ivancich [Tue, 17 Jan 2023 22:52:49 +0000 (17:52 -0500)]
Merge pull request #49463 from ivancich/wip-log-follow-bi-list

rgw: add logging to radosgw-admin bi list

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2 years agoMerge pull request #49648 from ivancich/wip-fix-fp-error
J. Eric Ivancich [Tue, 17 Jan 2023 22:51:38 +0000 (17:51 -0500)]
Merge pull request #49648 from ivancich/wip-fix-fp-error

rgw: fix FP error when calculating enteries per bi shard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #49181 from ljflores/wip-envlibrados-rocksdb-fix
Laura Flores [Tue, 17 Jan 2023 22:32:20 +0000 (16:32 -0600)]
Merge pull request #49181 from ljflores/wip-envlibrados-rocksdb-fix

qa/workunits/rados: skip running envlibrados rocksdb tests on ubuntu

2 years agoqa/workunits/rados: skip running envlibrados rocksdb tests on ubuntu 49181/head
Laura Flores [Thu, 1 Dec 2022 23:57:06 +0000 (17:57 -0600)]
qa/workunits/rados: skip running envlibrados rocksdb tests on ubuntu

This test passes on centos and rhel, but fails on ubuntu from an
invalid pointer. Since the envlibrados rocksdb tests are experimental
and don't have any actual users, we can just run them on rhel and
centos.

At the moment, the actual bug is not fully understood, but it was
decided that fixing it is low priority, and removing the test from
problematic distros is okay for the time being. This commit
is considered a workaround to the actual issue.

Related tracker: https://tracker.ceph.com/issues/57632
Signed-off-by: Laura Flores <lflores@redhat.com>
2 years agomgr/dashboard: fix bucket encryption checkbox 49765/head
Aashish Sharma [Tue, 17 Jan 2023 09:56:34 +0000 (15:26 +0530)]
mgr/dashboard: fix bucket encryption checkbox

Fixes: https://tracker.ceph.com/issues/58474
The encryption checkbox in the bucket creation form remains disabled after setting the vault authentication method as agent.

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2 years agomgr/dashboard: create authx users 47794/head
Pere Diaz Bou [Wed, 24 Aug 2022 17:28:38 +0000 (19:28 +0200)]
mgr/dashboard: create authx users

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
Co-authored-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #49533 from mkogan1/wip-fips-openssl3
Casey Bodley [Tue, 17 Jan 2023 13:12:54 +0000 (08:12 -0500)]
Merge pull request #49533 from mkogan1/wip-fips-openssl3

rgw: under fips & openssl 3.x allow md5 iusage in select rgw ops

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #49710 from cyx1231st/wip-seastore-cleanup-overwrite
Yingxin [Tue, 17 Jan 2023 08:29:24 +0000 (16:29 +0800)]
Merge pull request #49710 from cyx1231st/wip-seastore-cleanup-overwrite

crimson/os/seastore/object_data_handler: cleanup the overwrite path

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2 years agoMerge pull request #49743 from xxhdx1985126/wip-omap-fixes
Yingxin [Tue, 17 Jan 2023 08:28:43 +0000 (16:28 +0800)]
Merge pull request #49743 from xxhdx1985126/wip-omap-fixes

crimson/os/seastore/omap_manager: minor bugs fixes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agomgr/prometheus: export zero valued pg state metrics 49759/head
Avan Thakkar [Mon, 16 Jan 2023 12:41:06 +0000 (18:11 +0530)]
mgr/prometheus: export zero valued pg state metrics

Fixes: https://tracker.ceph.com/issues/58471
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
As per the Prometheus documentation, omitting zero metrics is not a best practice. The metric value for all PG_STATES should be initialized to zero.

2 years agoqa: add test_rbd_wnbd resize test
Lucian Petrut [Fri, 6 Jan 2023 10:58:00 +0000 (12:58 +0200)]
qa: add test_rbd_wnbd resize test

We're adding a test for the newly introduced live resize feature.
It will simply extend/shrink the image, wait for the new size to
be picked up and then run FIO tests to validate the resized image.

While at it, we're fixing two unrelated linter warnings:

  E275 missing whitespace after keyword

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agotest/crimson/seastore/test_omap_manager: keep test_omap_mappings consistent with... 49743/head
Xuehan Xu [Sat, 14 Jan 2023 03:05:12 +0000 (11:05 +0800)]
test/crimson/seastore/test_omap_manager: keep test_omap_mappings consistent with "rm_key_range" config

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2 years agocrimson/os/seastore/omap_manager: fix a capture-by-reference related issue
Xuehan Xu [Sat, 14 Jan 2023 01:09:45 +0000 (09:09 +0800)]
crimson/os/seastore/omap_manager: fix a capture-by-reference related issue

Variables that are not guaranteed to be available as long as the continuations
live shouldn't be captured by reference

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2 years agoMerge pull request #49725 from anthonyeleven/56521
zdover23 [Mon, 16 Jan 2023 03:50:17 +0000 (13:50 +1000)]
Merge pull request #49725 from anthonyeleven/56521

doc/releases: Clarify Quincy CephFS MDS upgrade process

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49747 from zdover23/wip-doc-2023-01-16-luks-versions
Anthony D'Atri [Mon, 16 Jan 2023 03:04:59 +0000 (22:04 -0500)]
Merge pull request #49747 from zdover23/wip-doc-2023-01-16-luks-versions

doc/ceph-volume: update LUKS docs

2 years agocrimson/os/seastore: add FIXME to consider pending extent with absolute paddr 49710/head
Yingxin Cheng [Wed, 11 Jan 2023 02:04:14 +0000 (10:04 +0800)]
crimson/os/seastore: add FIXME to consider pending extent with absolute paddr

During object data overwrite.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore/object_data_handler: cleanup the overwrite path
Yingxin Cheng [Wed, 11 Jan 2023 01:56:17 +0000 (09:56 +0800)]
crimson/os/seastore/object_data_handler: cleanup the overwrite path

* implement factory and type for extent_to_write_t.
* refactor overwrite_plan_t with explicit size calculations and
  defined operations.
* cleanup the overwrite path accordingly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #49716 from zdover23/wip-doc-2023-01-12-rbd-iscsi-initiator-linux...
zdover23 [Mon, 16 Jan 2023 01:32:02 +0000 (11:32 +1000)]
Merge pull request #49716 from zdover23/wip-doc-2023-01-12-rbd-iscsi-initiator-linux-prompts

doc/rbd: refine iscsi-initiator-linux.rbd

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/ceph-volume: update LUKS docs 49747/head
Zac Dover [Sun, 15 Jan 2023 21:46:49 +0000 (07:46 +1000)]
doc/ceph-volume: update LUKS docs

Remove references that claim that Ceph uses only LUKS version 1.

https://tracker.ceph.com/issues/58354

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49745 from zdover23/wip-doc-2023-01-15-start-documenting-ceph...
colemitchell [Sun, 15 Jan 2023 19:54:28 +0000 (14:54 -0500)]
Merge pull request #49745 from zdover23/wip-doc-2023-01-15-start-documenting-ceph-escaping-bold-characters-within-words

doc/start: add RST escape character rules for bold

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/rbd: format iscsi-initiator-linux.rbd better 49716/head
Zac Dover [Wed, 11 Jan 2023 16:00:29 +0000 (02:00 +1000)]
doc/rbd: format iscsi-initiator-linux.rbd better

Add prompts and clean up the lists in doc/rbd/iscsi-initiator-linux.rbd.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agodoc/start: add RST escape character rules for bold 49745/head
Zac Dover [Sun, 15 Jan 2023 10:45:28 +0000 (20:45 +1000)]
doc/start: add RST escape character rules for bold

Explain how to escape the bold notation (**) within words in RST.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #49736 from zdover23/wip-doc-2023-01-14-dev-basic-workflow-git...
colemitchell [Fri, 13 Jan 2023 21:47:52 +0000 (16:47 -0500)]
Merge pull request #49736 from zdover23/wip-doc-2023-01-14-dev-basic-workflow-git-commands

doc/dev: add git branch management commands

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/dev: add git branch management commands 49736/head
Zac Dover [Fri, 13 Jan 2023 20:48:57 +0000 (06:48 +1000)]
doc/dev: add git branch management commands

Add git branch deleting and search commands to the "Basic Workflow" page
of the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #48496 from adk3798/apply-spec-parsing
Adam King [Fri, 13 Jan 2023 19:00:27 +0000 (14:00 -0500)]
Merge pull request #48496 from adk3798/apply-spec-parsing

cephadm: only pull host info from applied spec, don't try to parse yaml

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #49024 from rkachach/fix_issue_58067
Adam King [Fri, 13 Jan 2023 17:29:58 +0000 (12:29 -0500)]
Merge pull request #49024 from rkachach/fix_issue_58067

mgr/cephadm: fixing mgr port shown in ceph orch ps

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge PR #49731 into main
Patrick Donnelly [Fri, 13 Jan 2023 17:28:05 +0000 (12:28 -0500)]
Merge PR #49731 into main

* refs/pull/49731/head:
tools/cephfs: fix traceback from missing envvar

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #48617 from adk3798/upgrade-status-help
Adam King [Fri, 13 Jan 2023 17:25:54 +0000 (12:25 -0500)]
Merge pull request #48617 from adk3798/upgrade-status-help

mgr/orchestrator: fix upgrade status help message

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #48384 from jmolmo/Rook_PDFetcher
Adam King [Fri, 13 Jan 2023 17:23:10 +0000 (12:23 -0500)]
Merge pull request #48384 from jmolmo/Rook_PDFetcher

mgr/rook: Device inventory

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2 years agoMerge pull request #48592 from adk3798/offline-upgrade
Adam King [Fri, 13 Jan 2023 17:01:41 +0000 (12:01 -0500)]
Merge pull request #48592 from adk3798/offline-upgrade

mgr/cephadm: improve offline host handling, mostly around upgrade

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #48574 from adk3798/debug-log-spam
Adam King [Fri, 13 Jan 2023 16:59:26 +0000 (11:59 -0500)]
Merge pull request #48574 from adk3798/debug-log-spam

mgr/cephadm: make logging refresh metadata to debug logs configurable

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #47383 from rhcs-dashboard/ceph-exporter-cephadm-changes
Adam King [Fri, 13 Jan 2023 16:54:01 +0000 (11:54 -0500)]
Merge pull request #47383 from rhcs-dashboard/ceph-exporter-cephadm-changes

exporter: cephadm changes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #49389 from adk3798/git-archive-fix
Adam King [Fri, 13 Jan 2023 15:31:21 +0000 (10:31 -0500)]
Merge pull request #49389 from adk3798/git-archive-fix

qa/tasks/cephadm.py: fix pulling cephadm from git.ceph.com

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #49726 from rhcs-dashboard/cephadm-e2e-fixes
Nizamudeen A [Fri, 13 Jan 2023 15:01:11 +0000 (20:31 +0530)]
Merge pull request #49726 from rhcs-dashboard/cephadm-e2e-fixes

mgr/dashboard: cephadm dashboard e2e fixes

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agorgw: Resolve sync stuck in previous gen 48898/head
Soumya Koduri [Mon, 26 Dec 2022 17:31:40 +0000 (12:31 -0500)]
rgw: Resolve sync stuck in previous gen

In case if any data log entries are missing for the older
generations, the sync server may not mark those shards as done
and can get stuck in that old gen for ever.

To avoid that, whenever a future gen entry is read, write the undone (shard,gen)
entry to error repo so that it can be processed and marked as done
and hence sync can progress eventually.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #49635 from rkachach/fix_issue_58378
Adam King [Fri, 13 Jan 2023 14:47:33 +0000 (09:47 -0500)]
Merge pull request #49635 from rkachach/fix_issue_58378

mgr/mgr_util: fixing some tox issues

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agorgw/multisite: return error from RGWLastCallerWinsCR() to track marker update failures
Shilpa Jagannath [Tue, 20 Dec 2022 15:37:55 +0000 (10:37 -0500)]
rgw/multisite: return error from RGWLastCallerWinsCR() to track marker update failures

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agorgw: Check for lost lease more often
Adam C. Emerson [Fri, 14 Oct 2022 16:49:03 +0000 (12:49 -0400)]
rgw: Check for lost lease more often

Flush the marker tracker and abort if we don't still have it.

Resolves: rhbz#2129718
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw/multisite: don't disable versioning on archive zone upon source bucket modifications
Shilpa Jagannath [Tue, 25 Oct 2022 14:52:48 +0000 (10:52 -0400)]
rgw/multisite: don't disable versioning on archive zone upon source bucket modifications

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agorgw/multisite: object should generate version id on archive zone, even when versioning
Shilpa Jagannath [Thu, 3 Nov 2022 02:23:52 +0000 (22:23 -0400)]
rgw/multisite: object should generate version id on archive zone, even when versioning
is disabled on primary

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agorgw/multisite: take the bucket lease before checking if bucket sync is enabled/disabled
Shilpa Jagannath [Fri, 11 Nov 2022 20:34:49 +0000 (15:34 -0500)]
rgw/multisite: take the bucket lease before checking if bucket sync is enabled/disabled

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agorgw/multisite: Add testcases for sync policy feature
Soumya Koduri [Wed, 18 Nov 2020 11:19:25 +0000 (16:49 +0530)]
rgw/multisite: Add testcases for sync policy feature

Added test cases for the various use-cases of multisite sync
policy feature listed in
https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agorgw: fix get sync policy handler if bucket_id is empty
Yehuda Sadeh [Tue, 1 Dec 2020 20:44:29 +0000 (12:44 -0800)]
rgw: fix get sync policy handler if bucket_id is empty

Fixes: https://tracker.ceph.com/issues/48416
bucket was passed in without bucket_id, now reading entrypoint
info if needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2 years agorgw: LatencyConcurrencyControl warns on very high latency
Adam C. Emerson [Wed, 12 Oct 2022 23:31:20 +0000 (19:31 -0400)]
rgw: LatencyConcurrencyControl warns on very high latency

Limited to only warn every five minutes.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agomgr/dashboard: cephadm dashboard e2e fixes 49726/head
Nizamudeen A [Thu, 12 Jan 2023 05:54:56 +0000 (11:24 +0530)]
mgr/dashboard: cephadm dashboard e2e fixes

Fixes: https://tracker.ceph.com/issues/58406
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #49417 from athanatos/sjust/backburner/wip-smp-option
Samuel Just [Fri, 13 Jan 2023 02:34:48 +0000 (18:34 -0800)]
Merge pull request #49417 from athanatos/sjust/backburner/wip-smp-option

crimson/osd/main: get full config before starting reactor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agorgw: Reduce data sync parallelism in response to RADOS lock latency
Adam C. Emerson [Tue, 11 Oct 2022 18:49:58 +0000 (14:49 -0400)]
rgw: Reduce data sync parallelism in response to RADOS lock latency

Lock latency in RGWContinuousLeaseCR gets high enough under load that
the locks end up timing out, leading to incorrect behavior.

Monitor lock latency and cut concurrent operations in half if it goes
above ten seconds.

Cut currency to one if it goes about twenty seconds.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: `SimpleRadosWriteAttrsCR` uses an async RADOS call
Adam C. Emerson [Wed, 24 Aug 2022 19:24:09 +0000 (15:24 -0400)]
rgw: `SimpleRadosWriteAttrsCR` uses an async RADOS call

Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: `SimpleRadosWriteCR` uses an async RADOS call
Adam C. Emerson [Wed, 24 Aug 2022 17:36:55 +0000 (13:36 -0400)]
rgw: `SimpleRadosWriteCR` uses an async RADOS call

Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: `SimpleRadosReadCR` uses an async RADOS call
Adam C. Emerson [Fri, 19 Aug 2022 22:58:21 +0000 (18:58 -0400)]
rgw: `SimpleRadosReadCR` uses an async RADOS call

Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agotools/cephfs: fix traceback from missing envvar 49731/head
Patrick Donnelly [Thu, 12 Jan 2023 21:12:52 +0000 (16:12 -0500)]
tools/cephfs: fix traceback from missing envvar

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agorgw: implement initial flight server functionality 48729/head
J. Eric Ivancich [Tue, 28 Jun 2022 19:35:18 +0000 (15:35 -0400)]
rgw: implement initial flight server functionality

Implements the ability for a flight to be created when the object is
retrieved by an S3 get.

Adds FlightServer abilities ListFlights, GetFlightInfo, GetSchema, and
DoGet. Adds an interface for a store for flight information and adds
an in-memory implemtation of it.

This code is functionality is early-stage and lacks some planned
efficiencies.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2 years agodoc/releases: Clarify Quincy CephFS MDS upgrade process 49725/head
Anthony D'Atri [Thu, 12 Jan 2023 05:11:50 +0000 (00:11 -0500)]
doc/releases: Clarify Quincy CephFS MDS upgrade process

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agoMerge pull request #49713 from rhcs-dashboard/fix-sse-daemon-name
Aashish Sharma [Thu, 12 Jan 2023 05:00:40 +0000 (10:30 +0530)]
Merge pull request #49713 from rhcs-dashboard/fix-sse-daemon-name

mgr/dashboard: rgw server side encryption daemon name fix

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #49540 from xxhdx1985126/wip-rebuild-missing-set
Yingxin [Thu, 12 Jan 2023 03:50:09 +0000 (11:50 +0800)]
Merge pull request #49540 from xxhdx1985126/wip-rebuild-missing-set

crimson/osd/pg: rebuild missing set when a new interval is created

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #48575 from liu-chunmei/multicore-cyanstore
Liu-Chunmei [Thu, 12 Jan 2023 00:40:33 +0000 (16:40 -0800)]
Merge pull request #48575 from liu-chunmei/multicore-cyanstore

crimson/os: cyanstore support multicore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #49712 from tchaikov/wip-packaging-in-setuptools
Laura Flores [Wed, 11 Jan 2023 23:50:47 +0000 (17:50 -0600)]
Merge pull request #49712 from tchaikov/wip-packaging-in-setuptools

mgr/prometheus: use vendored "packaging" instead

2 years agoMerge pull request #49719 from zdover23/wip-doc-2023-01-12-dev-developer-guide-basic...
colemitchell [Wed, 11 Jan 2023 21:33:02 +0000 (16:33 -0500)]
Merge pull request #49719 from zdover23/wip-doc-2023-01-12-dev-developer-guide-basic-workflow-miscellaneous-reorganization

doc/dev: add push.autoSetupRemote instructions

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2 years agodoc/dev: add push.autoSetupRemote instructions 49719/head
Zac Dover [Wed, 11 Jan 2023 20:24:55 +0000 (06:24 +1000)]
doc/dev: add push.autoSetupRemote instructions

Explain how to set up push.autoSetupRemote so that the origin branch is
properly set up when running "get push" without the need for the
"--set-upstream" option.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge pull request #48756 from Matan-B/wip-matanb-rollback-snap
Matan [Wed, 11 Jan 2023 16:18:43 +0000 (18:18 +0200)]
Merge pull request #48756 from Matan-B/wip-matanb-rollback-snap

crimson/osd: Support snapshot rollbacks

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #49568 from Matan-B/wip-matanb-crimson-modified-ranges
Matan [Wed, 11 Jan 2023 16:16:51 +0000 (18:16 +0200)]
Merge pull request #49568 from Matan-B/wip-matanb-crimson-modified-ranges

crimson/osd/*recovery_backend: Rename Push/Pull Info/Op

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
2 years agoMerge pull request #49715 from zdover23/wip-doc-2023-01-12-cephadm-host-management...
Anthony D'Atri [Wed, 11 Jan 2023 15:57:07 +0000 (10:57 -0500)]
Merge pull request #49715 from zdover23/wip-doc-2023-01-12-cephadm-host-management-osd-capitalization

doc/cephadm: s/osd/OSD/ where appropriate

2 years agoMerge PR #49711 into main
Venky Shankar [Wed, 11 Jan 2023 15:31:51 +0000 (21:01 +0530)]
Merge PR #49711 into main

* refs/pull/49711/head:
ceph.spec.in: remove the bcond_without cephfs_java option

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2 years agodoc/cephadm: s/osd/OSD/ where appropriate 49715/head
Zac Dover [Wed, 11 Jan 2023 15:12:24 +0000 (01:12 +1000)]
doc/cephadm: s/osd/OSD/ where appropriate

Capitalize the initialization "OSD" where it occurs in natural language
in cephadm/host-management.rst. This PR answers a request made by
Anthony D'Atri and seconded by Cole Mitchell in https://github.com/ceph/ceph/pull/49699#discussion_r1066171002.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2 years agomgr/prometheus: use vendored "packaging" instead 49712/head
Kefu Chai [Wed, 11 Jan 2023 04:12:17 +0000 (12:12 +0800)]
mgr/prometheus: use vendored "packaging" instead

instead of using the top-level "packaging" module, use the one
vendored by setuptools.

packaging python module provides versioning defined by PEP-440.
but python3-packaging is provided by CentOS8 powertools repo,
which is not enabled by default. and in CentOS9, this package
is provided by AppStream instead of BaseOS.

as prometheus mgr module is included by ceph-mgr-module-core,
it would be desirable if our user can install ceph-mgr-module-core
without enabling powertools or AppStream repo on a CentOS or
its derivative distros.

fortunately, setuptools vendors packaging module. and both
CentOS8 and CentOS9 provide python3-setuptools in their BaseOS
repos.

in this change, instead of using "packging" module, we use the
venderored one, which is in turn embedded in pkg_resources.
this python module is provided by python3-setuptools on CentOS
distros, and python3-pkg-resources on Debian and its derivatives.

the packaging recipes are updated accordingly to reflect the
new runtime dependency.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2 years agomgr/dashboard: rgw server side encryption daemon name fix 49713/head
Aashish Sharma [Wed, 11 Jan 2023 10:59:35 +0000 (16:29 +0530)]
mgr/dashboard: rgw server side encryption daemon name fix

The config values for the server side encryption currently are wrongly set to the mon daemon by default. This PR intends to fix this issue

Fixes: https://tracker.ceph.com/issues/58419
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2 years agorgw: `SimpleRadosReadAttrsCR` uses an async RADOS call
Adam C. Emerson [Wed, 24 Aug 2022 13:51:18 +0000 (09:51 -0400)]
rgw: `SimpleRadosReadAttrsCR` uses an async RADOS call

Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: Use `cls_version` for read/writes to global data sync status
Adam C. Emerson [Thu, 11 Aug 2022 22:36:06 +0000 (18:36 -0400)]
rgw: Use `cls_version` for read/writes to global data sync status

The `radosgw-admin data sync init` command does *not* use
`cls_version` and just overwrites.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: Reread sync status after acquiring lock in `RGWDataSyncCR`
Adam C. Emerson [Thu, 11 Aug 2022 22:19:59 +0000 (18:19 -0400)]
rgw: Reread sync status after acquiring lock in `RGWDataSyncCR`

If someone else got there first, we won't smash their work.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: Pull lock out of RGWInitDataSyncStatusCoroutine
Adam C. Emerson [Thu, 11 Aug 2022 21:54:58 +0000 (17:54 -0400)]
rgw: Pull lock out of RGWInitDataSyncStatusCoroutine

RGWDataSyncCR manages the lock instead, holding it through StateInit
and StateBuildingFullSyncMaps but releasing it by StateSync.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: RGWContinuousLeaseCR takes arguments by value and moves them
Adam C. Emerson [Thu, 11 Aug 2022 21:53:44 +0000 (17:53 -0400)]
rgw: RGWContinuousLeaseCR takes arguments by value and moves them

Since we were taking them by reference and copying before, this is
strictly better. Callers that give us an RValue can skip the copy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2 years agorgw: Get rid of some COOKIE_LEN preprocessor macros
Adam C. Emerson [Thu, 11 Aug 2022 21:52:17 +0000 (17:52 -0400)]
rgw: Get rid of some COOKIE_LEN preprocessor macros

Sticking random #defines everywhere is just atrocious style.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>