]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
23 months agoMerge pull request #52844 from lxbsz/wip-62202
Yuri Weinstein [Thu, 17 Aug 2023 15:35:22 +0000 (08:35 -0700)]
Merge pull request #52844 from lxbsz/wip-62202

pacific: mds: skip forwarding request if the session were removed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
23 months agoMerge pull request #52726 from kotreshhr/wip-62242-pacific
Yuri Weinstein [Thu, 17 Aug 2023 15:34:57 +0000 (08:34 -0700)]
Merge pull request #52726 from kotreshhr/wip-62242-pacific

pacific: mds: Fix the linkmerge assert check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
23 months agoMerge pull request #52682 from batrick/wip-62190-pacific
Yuri Weinstein [Thu, 17 Aug 2023 15:34:11 +0000 (08:34 -0700)]
Merge pull request #52682 from batrick/wip-62190-pacific

pacific: mds: update mdlog perf counters during replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
23 months agoMerge pull request #53002 from idryomov/wip-62437-pacific
Ilya Dryomov [Thu, 17 Aug 2023 10:22:11 +0000 (12:22 +0200)]
Merge pull request #53002 from idryomov/wip-62437-pacific

pacific: qa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test

Reviewed-by: Mykola Golub <mgolub@suse.com>
23 months agoMerge pull request #51039 from vshankar/tr-59003
Yuri Weinstein [Wed, 16 Aug 2023 16:22:15 +0000 (09:22 -0700)]
Merge pull request #51039 from vshankar/tr-59003

pacific: mgr/volumes: avoid returning -ESHUTDOWN back to cli

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
23 months agoMerge pull request #52873 from ifed01/wip-ifed-encrypted-ceph-volume-pac
Yuri Weinstein [Tue, 15 Aug 2023 20:59:37 +0000 (13:59 -0700)]
Merge pull request #52873 from ifed01/wip-ifed-encrypted-ceph-volume-pac

pacific: ceph_volume: support encrypted volumes for lvm new-db/new-wal/migrate commands

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
23 months agoMerge pull request #52212 from ifed01/wip-ifed-bluefs-cumulative-backports-pac
Yuri Weinstein [Tue, 15 Aug 2023 20:58:24 +0000 (13:58 -0700)]
Merge pull request #52212 from ifed01/wip-ifed-bluefs-cumulative-backports-pac

pacific: os/bluestore: cumulative bluefs backport

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
23 months agoMerge pull request #51773 from ifed01/wip-ifed-fix-bluefs-prealloc-pac
Yuri Weinstein [Tue, 15 Aug 2023 20:56:56 +0000 (13:56 -0700)]
Merge pull request #51773 from ifed01/wip-ifed-fix-bluefs-prealloc-pac

pacific: os/bluestore: proper override rocksdb::WritableFile::Allocate

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
23 months agoMerge pull request #51418 from ifed01/wip-ifed-fix-fit-to-fast-pac
Yuri Weinstein [Tue, 15 Aug 2023 20:55:52 +0000 (13:55 -0700)]
Merge pull request #51418 from ifed01/wip-ifed-fix-fit-to-fast-pac

pacific: os/bluestore: allow 'fit_to_fast' selector for single-volume osd

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
23 months agoqa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test 53002/head
Ilya Dryomov [Mon, 14 Aug 2023 11:16:59 +0000 (13:16 +0200)]
qa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test

The behavior of the class method changed in reef; the change was
backported to pacific and quincy.  An octopus test binary used against
pacific OSDs produces an expected failure:

    [ RUN      ] TestClsRbd.mirror_snapshot
    .../ceph-15.2.17/src/test/cls_rbd/test_cls_rbd.cc:2279: Failure
    Expected equality of these values:
      -85
      mirror_image_snapshot_unlink_peer(&ioctx, oid, 1, "peer2")
        Which is: 0
    [  FAILED  ] TestClsRbd.mirror_snapshot (6 ms)

Fixes: https://tracker.ceph.com/issues/62437
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
23 months agoMerge pull request #52468 from k0ste/wip-62031-pacific
Yuri Weinstein [Fri, 11 Aug 2023 15:28:26 +0000 (08:28 -0700)]
Merge pull request #52468 from k0ste/wip-62031-pacific

pacific: mon/ConfigMonitor: update crush_location from osd entity

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
23 months agoMerge pull request #51812 from NitzanMordhai/wip-61488-pacific
Yuri Weinstein [Fri, 11 Aug 2023 15:27:46 +0000 (08:27 -0700)]
Merge pull request #51812 from NitzanMordhai/wip-61488-pacific

pacific: pybind/argparse: blocklist ip validation

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
23 months agoMerge pull request #51382 from RaminNietzsche/wip-ifed-fix-require-osd-release-to...
Igor Fedotov [Fri, 11 Aug 2023 07:42:51 +0000 (10:42 +0300)]
Merge pull request #51382 from RaminNietzsche/wip-ifed-fix-require-osd-release-to-pacific

pacific: mon: avoid exception when setting require-osd-release more than 2 versions up.

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
23 months agoMerge pull request #52500 from lxbsz/wip-62040
Venky Shankar [Wed, 9 Aug 2023 12:05:58 +0000 (17:35 +0530)]
Merge pull request #52500 from lxbsz/wip-62040

pacific: client: do not send metrics until the MDS rank is ready

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52654 from joscollin/wip-62177-pacific
Venky Shankar [Wed, 9 Aug 2023 12:05:32 +0000 (17:35 +0530)]
Merge pull request #52654 from joscollin/wip-62177-pacific

pacific: qa: fix cephfs-mirror unwinding and 'fs volume create/rm' order

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52505 from lxbsz/wip-62012
Venky Shankar [Wed, 9 Aug 2023 04:32:01 +0000 (10:02 +0530)]
Merge pull request #52505 from lxbsz/wip-62012

pacific: client: wait rename to finish

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52506 from lxbsz/wip-61983
Venky Shankar [Wed, 9 Aug 2023 04:31:32 +0000 (10:01 +0530)]
Merge pull request #52506 from lxbsz/wip-61983

pacific: client: force sending cap revoke ack always

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52513 from joscollin/wip-62055-pacific
Venky Shankar [Wed, 9 Aug 2023 04:30:38 +0000 (10:00 +0530)]
Merge pull request #52513 from joscollin/wip-62055-pacific

pacific: mds: MDLog::_recovery_thread: handle the errors gracefully

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52499 from lxbsz/wip-62043
Venky Shankar [Wed, 9 Aug 2023 04:29:58 +0000 (09:59 +0530)]
Merge pull request #52499 from lxbsz/wip-62043

pacific: client: trigger to flush the buffer when making snapshot

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52397 from mchangir/wip-61961-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:20:47 +0000 (11:20 -0700)]
Merge pull request #52397 from mchangir/wip-61961-pacific

pacific: mon: block osd pool mksnap for fs pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52304 from lxbsz/wip-61798
Yuri Weinstein [Tue, 8 Aug 2023 18:20:19 +0000 (11:20 -0700)]
Merge pull request #52304 from lxbsz/wip-61798

pacific: client: only wait for write MDS OPs when unmounting

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52244 from batrick/wip-61426-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:19:46 +0000 (11:19 -0700)]
Merge pull request #52244 from batrick/wip-61426-pacific

pacific: mon/MDSMonitor: ignore extraneous up:boot messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52240 from batrick/wip-61414-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:19:10 +0000 (11:19 -0700)]
Merge pull request #52240 from batrick/wip-61414-pacific

pacific: mon/MDSMonitor: do not propose on error in prepare_update

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52237 from batrick/wip-59372-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:18:36 +0000 (11:18 -0700)]
Merge pull request #52237 from batrick/wip-59372-pacific

pacific: qa: wait for MDSMonitor tick to replace daemons

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52233 from batrick/wip-61411-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:18:01 +0000 (11:18 -0700)]
Merge pull request #52233 from batrick/wip-61411-pacific

pacific: mon/MDSMonitor: check fscid in pending exists in current

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52230 from batrick/wip-61692-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:17:22 +0000 (11:17 -0700)]
Merge pull request #52230 from batrick/wip-61692-pacific

pacific:  mon/MDSMonitor: batch last_metadata update with pending

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52125 from joscollin/wip-61734-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:16:53 +0000 (11:16 -0700)]
Merge pull request #52125 from joscollin/wip-61734-pacific

pacific: mds: display sane hex value (0x0) for empty feature bit

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52075 from joscollin/wip-61696-pacific
Yuri Weinstein [Tue, 8 Aug 2023 18:16:13 +0000 (11:16 -0700)]
Merge pull request #52075 from joscollin/wip-61696-pacific

pacific: debian: install cephfs-mirror systemd unit files and man page

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51508 from lxbsz/wip-59706
Yuri Weinstein [Tue, 8 Aug 2023 18:15:29 +0000 (11:15 -0700)]
Merge pull request #51508 from lxbsz/wip-59706

pacific: mds: do not take the ino which has been used

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoceph-volume: make raw prepare use encryption_units.prepare_dmcrypt 52873/head
Igor Fedotov [Mon, 7 Aug 2023 09:04:10 +0000 (12:04 +0300)]
ceph-volume: make raw prepare use encryption_units.prepare_dmcrypt

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 2c3477a69e2e01e999ff23ecf4a6508c87c340de)

23 months agotest/ceph_volume: add UTs for encrypted volumes migration
Igor Fedotov [Thu, 9 Mar 2023 15:06:09 +0000 (18:06 +0300)]
test/ceph_volume: add UTs for encrypted volumes migration

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit f69316f627014b720f6b89dcb96c29cd7480cf88)

23 months agoceph-volume: close encrypted device using name not path when zapping
Igor Fedotov [Thu, 9 Mar 2023 13:30:56 +0000 (16:30 +0300)]
ceph-volume: close encrypted device using name not path when zapping

Just a tiny cleanup.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit ebc335def0b32d1ecfe93fcfa286ddf460f136a1)

23 months agoceph-volume: close encrypted volumes on deactivate
Igor Fedotov [Thu, 9 Mar 2023 13:29:55 +0000 (16:29 +0300)]
ceph-volume: close encrypted volumes on deactivate

Fixes: https://tracker.ceph.com/issues/58943
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit d8f163d1f55d8234824e57fc6f851db9292fa972)

23 months agotest/ceph-volume: add UT for adding encrypted DB/WAL lvm.
Igor Fedotov [Tue, 7 Mar 2023 17:52:25 +0000 (20:52 +0300)]
test/ceph-volume: add UT for adding encrypted DB/WAL lvm.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 9930225391eccd27d028592a90e677c86f01b59c)

23 months agoceph-volume: support creating/migrating encrypted lvm volumes
Igor Fedotov [Tue, 7 Mar 2023 16:53:57 +0000 (19:53 +0300)]
ceph-volume: support creating/migrating encrypted lvm volumes

Fixes: https://tracker.ceph.com/issues/55167
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit b45d9b79aa079f3cfe227c74c1df3a3bfad4c7c9)

23 months agoMerge pull request #51249 from k0ste/wip-52791-pacific
Yuri Weinstein [Mon, 7 Aug 2023 20:24:15 +0000 (13:24 -0700)]
Merge pull request #51249 from k0ste/wip-52791-pacific

pacific: common/TrackedOp: fix osd reboot optracker coredump

Reviewed-by: Laura Flores <lflores@redhat.com>
23 months agoos/bluestore: introduce a cooldown period for failed BlueFS allocations. 52212/head
Igor Fedotov [Fri, 11 Nov 2022 14:31:19 +0000 (17:31 +0300)]
os/bluestore: introduce a cooldown period for failed BlueFS allocations.

When using bluefs_shared_alloc_size one might get a long-lasting state when
that large chunks are not available any more and fallback to shared
device min alloc size occurs. The introduced cooldown is intended to
prevent repetitive allocation attempts with bluefs_shared_alloc_size for
a while. The rationale is to eliminate performance penalty these failing
attempts might cause.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit e52bcc852ce51ab99138420f9069e2f59e1cb706)

 Conflicts:
src/common/options/global.yaml.in
 (legacy options declarations, no yamls in pacific)

23 months agoMerge pull request #52704 from rhcs-dashboard/wip-62237-pacific
Nizamudeen A [Mon, 7 Aug 2023 04:38:18 +0000 (10:08 +0530)]
Merge pull request #52704 from rhcs-dashboard/wip-62237-pacific

pacific: mgr/dashboard: allow PUT in CORS

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
23 months agomds: skip forwarding request if the session were removed 52844/head
Xiubo Li [Mon, 10 Jul 2023 03:48:55 +0000 (11:48 +0800)]
mds: skip forwarding request if the session were removed

When forwarding the requests, the corresponding session could be
already closed.

https://tracker.ceph.com/issues/60625
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit d0bfbbea44e22fc545363cd6af47d28e18e353b0)

23 months agoMerge pull request #52790 from adamemerson/wip-62097-pacific
Ilya Dryomov [Thu, 3 Aug 2023 19:47:45 +0000 (21:47 +0200)]
Merge pull request #52790 from adamemerson/wip-62097-pacific

pacific: build: Remove ceph-libboost* packages in install-deps

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
23 months agomgr: fix some flake8 complaints 52790/head
Nizamudeen A [Mon, 31 Jul 2023 09:20:57 +0000 (14:50 +0530)]
mgr: fix some flake8 complaints

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit d199782fb5f3c22f1cd5fda6ddf1d64f40a92726)

Conflicts:
src/pybind/mgr/crash/module.py imports were not present

23 months agobuild: install-deps.sh installs system boost on Jammy 52768/head
Adam Emerson [Thu, 20 Jul 2023 01:03:44 +0000 (21:03 -0400)]
build: install-deps.sh installs system boost on Jammy

Since on Jammy system boost is new enough for Pacific and we don't have
Jammy packages for older boost (we only have those for Bionic), just
install the system packages rather than fetching ceph-libboost.

No analogous commit exists in main as while main's Jammy case installs
ceph-libboost, we just need a system package here.

Fixes: https://tracker.ceph.com/issues/62103
Signed-off-by: Adam Emerson <aemerson@redhat.com>
23 months agobuild: Remove old ceph-libboost* packages in install-deps
Adam Emerson [Wed, 19 Jul 2023 21:12:08 +0000 (17:12 -0400)]
build: Remove old ceph-libboost* packages in install-deps

Here, we extract `clean_boost_on_ubuntu()` and call it before other
installs on Debian distributions so that if we install a system boost,
a potentially newer `ceph-libboost` won't get in the way.

As the sources.list.d being removed in the original cleanup code isn't
the one we're currently installing in the install code, add a removal
for the currently used source, then do apt-update so packages from the
removed source are no longer included as available.

Two subsidiary dev packages from conflicting boost libraries can be
installed, but it leaves apt in an inconsistent state. To clean this
up, add `--fix-missing` to the removal line and call
`clean_boost_on_ubuntu()` before other uses of apt.

Fixes: https://tracker.ceph.com/issues/62097
Signed-off-by: Adam Emerson <aemerson@redhat.com>
(cherry picked from commit 0c3f511e14af639b6509e69b889258b2f718f8fd)

Conflicts:
install-deps.sh
 - Different boost version for Pacific than Squid.
 - ci_debug does not exist in Pacific
 - whitespace
 - No INSTALL_EXTRA

Fixes: https://tracker.ceph.com/issues/62103
Signed-off-by: Adam Emerson <aemerson@redhat.com>
23 months agotest/store_test: adjust spillover cases to reflect updated WAL usage. 51773/head
Igor Fedotov [Wed, 2 Aug 2023 13:49:51 +0000 (16:49 +0300)]
test/store_test: adjust spillover cases to reflect updated WAL usage.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
23 months agoMerge pull request #52159 from cbodley/wip-57643-pacific
Yuri Weinstein [Tue, 1 Aug 2023 15:16:08 +0000 (11:16 -0400)]
Merge pull request #52159 from cbodley/wip-57643-pacific

pacific: rgw: avoid string_view to temporary in RGWBulkUploadOp

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
23 months agoMerge pull request #49943 from NitzanMordhai/wip-58611-pacific
Yuri Weinstein [Tue, 1 Aug 2023 15:13:21 +0000 (11:13 -0400)]
Merge pull request #49943 from NitzanMordhai/wip-58611-pacific

pacific: Wip nitzan fixing few rados/test.sh

Reviewed-by: Laura Flores <lflores@redhat.com>
23 months agoMerge pull request #49521 from NitzanMordhai/wip-58314-pacific
Yuri Weinstein [Tue, 1 Aug 2023 15:12:40 +0000 (11:12 -0400)]
Merge pull request #49521 from NitzanMordhai/wip-58314-pacific

pacific: src/valgrind.supp: Adding know leaks unrelated to ceph

Reviewed-by: Laura Flores <lflores@redhat.com>
23 months agoos/bluestore: proper override rocksdb::WritableFile::Allocate
Igor Fedotov [Wed, 1 Mar 2023 13:04:23 +0000 (16:04 +0300)]
os/bluestore: proper override rocksdb::WritableFile::Allocate

Failing to do that makes BlueFS file frgmentation higher.

Fixes: https://tracker.ceph.com/issues/58966
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 2d8b9e9ed58bca4917b90ea5eaa43264a89dff75)

23 months agomds: Fix the linkmerge assert check 52726/head
Kotresh HR [Tue, 6 Jun 2023 07:39:00 +0000 (13:09 +0530)]
mds: Fix the linkmerge assert check

Let's say there is a hardlink created as below.

touch file1
ln file1 hl_file1

In this case 'file1' holds the primary inode and 'hl_file' holds
the remote inode (holds primary inode number and other required info).
Now, if the 'file1' is deleted first, it's moved to a stray directory
and can't be deleted because the hardlink 'hl_file1' still exists
which requires primary inode. So on straydn eval, this primary
inode is linked to 'hl_file1 and remote inode is removed. This is called
the linkmerge/stray reintegration. So in the linkmerge case, the srcdnl
is primary straydn and the destdnl is the remote.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
Fixes: https://tracker.ceph.com/issues/61879
(cherry picked from commit 85279ac657832fd643ee2cc5f1f6a27f98e099f6)

23 months agomgr/dashboard: allow PUT in CORS 52704/head
Nizamudeen A [Tue, 25 Jul 2023 09:21:40 +0000 (14:51 +0530)]
mgr/dashboard: allow PUT in CORS

Fixes: https://tracker.ceph.com/issues/62222
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 8da2a2c0e296a94a2bfab6ab68480d2e33e7d84e)

23 months agomds: update mdlog perf counters during replay 52682/head
Patrick Donnelly [Fri, 30 Jun 2023 16:44:15 +0000 (12:44 -0400)]
mds: update mdlog perf counters during replay

Fixes: https://tracker.ceph.com/issues/61864
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 48e3b51151fe94217bb4064b8fc9701e3536730c)

23 months agoqa: fix cephfs-mirror unwinding and 'fs volume create/rm' order 52654/head
Jos Collin [Mon, 24 Jul 2023 08:46:52 +0000 (14:16 +0530)]
qa: fix cephfs-mirror unwinding and 'fs volume create/rm' order

* Fixes the 'fs volume create' happens before the cephfs-mirror daemon start.
* Fixes the 'fs volume rm' happen only after the cephfs-mirror daemon unwinding.
  - This prevents the issue of mirror-daemon not returning from a libcephfs call, as
    the volumes were deleted during the cephfs_mirror_thrash ing.

Fixes: https://tracker.ceph.com/issues/61182
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit b9a1a3cdf9770bcb27d6e08ddbc059f01674f4b8)

23 months agomds: MDLog::_recovery_thread: handle the errors gracefully 52513/head
Jos Collin [Fri, 23 Jun 2023 06:16:26 +0000 (11:46 +0530)]
mds: MDLog::_recovery_thread: handle the errors gracefully

A write fails if the MDS is already blocklisted due to the 'fs fail' issued by the qa tests.
Handle those write failures gracefully, even when the MDS is stopping.

Fixes: https://tracker.ceph.com/issues/61201
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit d562905dcfb5b8a45ce7042c543720ef8b0fa05b)

23 months agoMerge pull request #50814 from dparmar18/wip-59261-pacific
Venky Shankar [Wed, 26 Jul 2023 08:07:32 +0000 (13:37 +0530)]
Merge pull request #50814 from dparmar18/wip-59261-pacific

pacific: mds: fix stray evaluation using scrub and introduce new option

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51686 from lxbsz/wip-61346
Venky Shankar [Wed, 26 Jul 2023 06:25:26 +0000 (11:55 +0530)]
Merge pull request #51686 from lxbsz/wip-61346

pacific: mds: wait reintegrate to finish when unlinking

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51699 from dparmar18/wip-61202-pacific
Yuri Weinstein [Tue, 25 Jul 2023 15:20:07 +0000 (11:20 -0400)]
Merge pull request #51699 from dparmar18/wip-61202-pacific

pacific: MDS imported_inodes metric is not updated.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51687 from vshankar/tr-59725
Yuri Weinstein [Tue, 25 Jul 2023 15:19:34 +0000 (11:19 -0400)]
Merge pull request #51687 from vshankar/tr-59725

pacific: mds: allow unlink from lost+found directory

Reviewed-by: Xiubo Li <xiubli@redhat.com>
23 months agoMerge pull request #51610 from vshankar/tr-59721
Yuri Weinstein [Tue, 25 Jul 2023 15:17:49 +0000 (11:17 -0400)]
Merge pull request #51610 from vshankar/tr-59721

pacific: qa: run scrub post file system recovery

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51609 from vshankar/tr-61235
Yuri Weinstein [Tue, 25 Jul 2023 15:17:01 +0000 (11:17 -0400)]
Merge pull request #51609 from vshankar/tr-61235

pacific: mds: rdlock_path_xlock_dentry supports returning auth target inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51353 from joscollin/wip-59482-pacific
Yuri Weinstein [Tue, 25 Jul 2023 15:16:17 +0000 (11:16 -0400)]
Merge pull request #51353 from joscollin/wip-59482-pacific

pacific: cephfs-top: check the minimum compatible python version

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51247 from k0ste/wip-52442-pacific
Yuri Weinstein [Tue, 25 Jul 2023 15:15:50 +0000 (11:15 -0400)]
Merge pull request #51247 from k0ste/wip-52442-pacific

pacific: client: do not dump mds twice in Inode::dump()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #51246 from mchangir/wip-59017-pacific
Yuri Weinstein [Tue, 25 Jul 2023 15:15:21 +0000 (11:15 -0400)]
Merge pull request #51246 from mchangir/wip-59017-pacific

pacific: mgr/snap_schedule: add debug log for paths failing snapshot creation

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #50737 from joscollin/wip-58984-pacific
Yuri Weinstein [Tue, 25 Jul 2023 15:12:43 +0000 (11:12 -0400)]
Merge pull request #50737 from joscollin/wip-58984-pacific

pacific: cephfs-top: navigate to home screen when no fs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
23 months agoMerge pull request #52323 from batrick/wip-61900-pacific
Yuri Weinstein [Mon, 24 Jul 2023 15:03:13 +0000 (11:03 -0400)]
Merge pull request #52323 from batrick/wip-61900-pacific

pacific: pybind: drop GIL during library callouts

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
23 months agoMerge pull request #52287 from idryomov/wip-59641-pacific
Yuri Weinstein [Mon, 24 Jul 2023 15:02:09 +0000 (11:02 -0400)]
Merge pull request #52287 from idryomov/wip-59641-pacific

pacific: librbd/managed_lock/GetLockerRequest: Fix no valid lockers case

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
23 months agoMerge pull request #52285 from idryomov/wip-61616-pacific
Yuri Weinstein [Mon, 24 Jul 2023 15:01:27 +0000 (11:01 -0400)]
Merge pull request #52285 from idryomov/wip-61616-pacific

pacific: librbd: avoid object map corruption in snapshots taken under I/O

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2 years agoMerge pull request #52063 from guits/wip-61154-pacific
Guillaume Abrioux [Wed, 19 Jul 2023 08:03:11 +0000 (10:03 +0200)]
Merge pull request #52063 from guits/wip-61154-pacific

pacific: ceph-volume: fix a bug in `get_lvm_fast_allocs()` (batch)

2 years agoMerge pull request #52080 from guits/wip-61698-pacific
Guillaume Abrioux [Wed, 19 Jul 2023 08:01:21 +0000 (10:01 +0200)]
Merge pull request #52080 from guits/wip-61698-pacific

pacific: ceph-volume: set lvm membership for mpath type devices

2 years agodoc: users now need to provide scrub_mdsdir and recursive flags 50814/head
Dhairya Parmar [Mon, 22 May 2023 12:00:27 +0000 (17:30 +0530)]
doc: users now need to provide scrub_mdsdir and recursive flags
for mdsdir scrub at CephFS root.

Fixes: https://tracker.ceph.com/issues/59350
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 93dfc111bf7e11b3122d2a2df0346b341164dd08)

2 years agoqa: add recursive flag to test_flag_scrub_mdsdir
Dhairya Parmar [Mon, 22 May 2023 11:55:38 +0000 (17:25 +0530)]
qa: add recursive flag to test_flag_scrub_mdsdir

Code has been changed, in order to scrub ~mdsdir at root,
recursive flag also needs to be provided along with
scrub_mdsdir.

Fixes: https://tracker.ceph.com/issues/59350
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit e40ca408a1343754b80ba1e88e0b669689ada164)

2 years agomds: remove code to bypass dumping empty header scrub info
Dhairya Parmar [Mon, 22 May 2023 10:37:34 +0000 (16:07 +0530)]
mds: remove code to bypass dumping empty header scrub info

Previously when ~mdsdir was scrubbed at CephFS root, it's header
was kept empty, thus it became necessary to not dump it's values
for 'scrub status'. Now since both the scrubs(~mdsdir and root)
run under the same header, this code is no more needed.

Fixes: https://tracker.ceph.com/issues/59350
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 548af10f1057fe3fb46e23887b012a664b45b7f8)

2 years agomds: dump_values no more needed
Dhairya Parmar [Mon, 22 May 2023 10:36:24 +0000 (16:06 +0530)]
mds: dump_values no more needed

Previouly, two individual scrubs were initiated to scrub ~mdsdir
at root where the ~mdsdir scrub wasn't provided any tag thus, it
was necessary to not dump it's values for output of 'scrub start'.
Now since mdsdir and root scrub run under single header, there is
no need for this anymore, thus removing this redundant code.

Fixes: https://tracker.ceph.com/issues/59350
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 2f06feea33c88798e4e9d654bbff8b05e77f0681)

2 years agomds: enqueue ~mdsdir at the time of enqueing root
Dhairya Parmar [Mon, 22 May 2023 07:04:51 +0000 (12:34 +0530)]
mds: enqueue ~mdsdir at the time of enqueing root

This would avoid the need to run individual scrubs for
~mdsdir and root, i.e. run both the scrubs under the
same header, this also helps to avoid edge case where
in case ~mdsdir is huge and it's taking time to scrub it,
the scrub status would report something like this until
root inodes kick in:

{
    "status": "scrub active (757 inodes in the stack)",
    "scrubs": {}
}

Fixes: https://tracker.ceph.com/issues/59350
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit f85b22818b474911cfdc15e7c6be1d2979939888)

2 years agoclient: force sending cap revoke ack always 52506/head
Xiubo Li [Wed, 28 Jun 2023 13:59:53 +0000 (21:59 +0800)]
client: force sending cap revoke ack always

If just before the revoke request, which will increase the 'seq', is
sent out the clients released the corresponding caps and sent out
the cap update request with old 'seq', the mds will miss checking
the seqs and calculating the caps.

We should always send an ack for revoke requests.

Fixes: https://tracker.ceph.com/issues/61782
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 09a2e33c4aca6e72d75e2527ab5f467576d55dc8)

2 years agoclient: wait rename to finish 52505/head
Xiubo Li [Sat, 10 Apr 2021 04:52:24 +0000 (12:52 +0800)]
client: wait rename to finish

In rare case during the rename if another thread tries to lookup the
dst dentry, it may get an inconsistent result that both src dentry
and dst dentry will link to the same inode at the same time.

Will wait the rename to finish and try it again.

Fixes: https://tracker.ceph.com/issues/49912
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 11fb16df06ad61e228c17feedf00dfb85f0477d5)

Conflicts:
src/client/Client.h: a minor conflict.

2 years agoclient: do not send metrics until the MDS rank is ready 52500/head
Xiubo Li [Thu, 1 Jun 2023 12:00:01 +0000 (20:00 +0800)]
client: do not send metrics until the MDS rank is ready

In some cases when there are a lot of clients and these clients
have a lots of known requests need to replay too, the metrics
requests will be dropped by the MDS because the MDS is still in
the clientreplay state, and also the useless metric requests will
slow down MDS.

Fixes: https://tracker.ceph.com/issues/61523
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit a423e17d5bbd095af4472c087dbbc6ac0c5e9e61)

2 years agoclient: clean up the queue_cap_snap() code to remove unnecessary check 52499/head
Xiubo Li [Mon, 8 May 2023 11:34:19 +0000 (19:34 +0800)]
client: clean up the queue_cap_snap() code to remove unnecessary check

If the 'dirty' is true then 'dirty & CEPH_CAP_ANY_WR' will always be
true.

Fixes: https://tracker.ceph.com/issues/59343
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit cebb5a61c507e05fdd391ee1c569e5a3a0860c2a)

2 years agoclient: always add one new capsnap if Fb is used and Fw is not used
Xiubo Li [Mon, 8 May 2023 05:48:43 +0000 (13:48 +0800)]
client: always add one new capsnap if Fb is used and Fw is not used

If we set the 'writing' to 1 when the 'Fb' caps is used then later
if we have any dirty caps it will be skipped and will reuse the
existing capsnap, which is incorrect.

At the same time trigger to flush the buffer when making snapshot
and if the Fb is being used.

Fixes: https://tracker.ceph.com/issues/59343
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 6262351f7da9413d8306a6f1765ecab1f0ab912e)

2 years agoMerge pull request #52477 from rhcs-dashboard/fedora-url-pacific
Pere Diaz Bou [Mon, 17 Jul 2023 09:41:19 +0000 (11:41 +0200)]
Merge pull request #52477 from rhcs-dashboard/fedora-url-pacific

pacific: mgr/dashboard: Fix broken Fedora image URL

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: zmc <NOT@FOUND>
2 years agomgr/dashboard: Fix broken Fedora image URL 52477/head
Zack Cerza [Thu, 29 Sep 2022 19:44:46 +0000 (13:44 -0600)]
mgr/dashboard: Fix broken Fedora image URL

This mirror apparently stopped working, breaking jobs that use this
script.

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 3f32ab0fb3a54174a8a57681243cd944056c6ffb)

2 years agomon/ConfigMonitor: update crush_location from osd entity 52468/head
Didier Gazen [Thu, 15 Jun 2023 13:57:24 +0000 (15:57 +0200)]
mon/ConfigMonitor: update crush_location from osd entity

For osd entities, crush_location needs to refer to osd's parent
(host) so that 'ceph config set' using osd/host mask can work.

Fixes: https://tracker.ceph.com/issues/48750
Signed-off-by: Didier Gazen <didier.gazen@aero.obs-mip.fr>
(cherry picked from commit 5df2a597a5b90ed492b2cbc3599f2cfdababeb12)

2 years agoMerge pull request #52389 from rhcs-dashboard/cleanup-script-pacific
Nizamudeen A [Fri, 14 Jul 2023 12:06:14 +0000 (17:36 +0530)]
Merge pull request #52389 from rhcs-dashboard/cleanup-script-pacific

pacific: mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #50321 from ifed01/wip-ifed-log-before-assert-in-avl-pac
Igor Fedotov [Thu, 13 Jul 2023 18:50:16 +0000 (21:50 +0300)]
Merge pull request #50321 from ifed01/wip-ifed-log-before-assert-in-avl-pac

pacific: os/bluestore: allocator's cumulative backport

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2 years agoMerge pull request #51510 from Matan-B/wip-61149-pacific
Yuri Weinstein [Wed, 12 Jul 2023 20:44:32 +0000 (16:44 -0400)]
Merge pull request #51510 from Matan-B/wip-61149-pacific

pacific: OSD: Fix check_past_interval_bounds()

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoqa: add test to verify blocking of osd pool mksnap for fs pools 52397/head
Milind Changire [Wed, 26 Apr 2023 05:11:24 +0000 (10:41 +0530)]
qa: add test to verify blocking of osd pool mksnap for fs pools

Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit ab64bfaaf978cb9a14ff0520d4dc51ea32e1e93c)

2 years agomon: block osd pool mksnap for fs pools
Milind Changire [Mon, 8 May 2023 07:52:12 +0000 (13:22 +0530)]
mon: block osd pool mksnap for fs pools

Commit 23db15d5c2b disabled pool snaps for the rados mksnap path. But
ceph osd pool mksnap was an alternate way that pool snaps could be
created.
This commit disables pool snaps via this alternate path as well.

NOTE:
Pool-level snaps and fs-level snaps can't co-exist since snap IDs are
likely to clash between the two different mechanisms and can result in
unintentional data loss when either of the snaps are deleted.

Fixes: https://tracker.ceph.com/issues/59552
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 6203fe15289d27bf02441fd413a40371cb0bc516)

2 years agomgr/dashboard: move cephadm e2e cleanup to jenkins job config 52389/head
Nizamudeen A [Fri, 7 Jul 2023 10:46:00 +0000 (16:16 +0530)]
mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Follows: https://github.com/ceph/ceph-build/pull/2145

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a407f0b7834585c087001059d6cec613e524538b)

2 years agoMerge pull request #52117 from cbodley/wip-61730-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:42:41 +0000 (17:42 -0400)]
Merge pull request #52117 from cbodley/wip-61730-pacific

pacific: librados: aio operate functions can set times

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51803 from NitzanMordhai/wip-61448-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:41:59 +0000 (17:41 -0400)]
Merge pull request #51803 from NitzanMordhai/wip-61448-pacific

pacific: test: correct osd pool default size

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
2 years agoMerge pull request #51799 from NitzanMordhai/wip-61450-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:41:23 +0000 (17:41 -0400)]
Merge pull request #51799 from NitzanMordhai/wip-61450-pacific

pacific: test: monitor thrasher wait until quorum

Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2 years agoMerge pull request #51604 from Svelar/pacific-vstart-mgr-status
Yuri Weinstein [Wed, 5 Jul 2023 21:40:47 +0000 (17:40 -0400)]
Merge pull request #51604 from Svelar/pacific-vstart-mgr-status

pacific: vstart: check mgr status after starting mgr

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #51316 from rosinL/wip-backport-42082-to-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:38:19 +0000 (17:38 -0400)]
Merge pull request #51316 from rosinL/wip-backport-42082-to-pacific

pacific: common: Use double instead of long double to improve performance

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #51315 from rosinL/wip-backport-41393-to-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:36:40 +0000 (17:36 -0400)]
Merge pull request #51315 from rosinL/wip-backport-41393-to-pacific

pacific: common/crc32c_aarch64: fix crc32c unittest failed on aarch64

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #51254 from k0ste/wip-52892-pacific
Yuri Weinstein [Wed, 5 Jul 2023 21:35:43 +0000 (17:35 -0400)]
Merge pull request #51254 from k0ste/wip-52892-pacific

pacific: tools/ceph-kvstore-tool: fix segfaults when repair the rocksdb

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2 years agoMerge pull request #50980 from batrick/i59296
Yuri Weinstein [Wed, 5 Jul 2023 21:35:04 +0000 (17:35 -0400)]
Merge pull request #50980 from batrick/i59296

pacific: MgrMonitor: batch commit OSDMap and MgrMap mutations

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #52133 from rzarzynski/wip-61740-pacific
Radoslaw Zarzynski [Wed, 5 Jul 2023 18:20:35 +0000 (20:20 +0200)]
Merge pull request #52133 from rzarzynski/wip-61740-pacific

pacific: mon/MonClient: before complete auth with error, reopen session

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agopybind/rbd: drop GIL when calling into librbd 52323/head
Ilya Dryomov [Mon, 3 Jul 2023 11:34:30 +0000 (13:34 +0200)]
pybind/rbd: drop GIL when calling into librbd

This was missing for rbd_mirror_peer_site_add() and
rbd_get_data_pool_id().

While at it, add a test for data_pool_id().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1a60f66b0a13089c789e2e056b925ac6fef75021)

Conflicts:
src/test/pybind/test_rbd.py: trivial import conflict

2 years agopybind/cephfs: drop gil during cephfs callouts
Patrick Donnelly [Sun, 2 Jul 2023 16:05:08 +0000 (12:05 -0400)]
pybind/cephfs: drop gil during cephfs callouts

This has disastorous consequences including the possibility of deadlock.
In the best case, you have the rmdir holding the GIL until the MDS
responds!

Fixes: https://tracker.ceph.com/issues/61869
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5d249a4c978198f1e8974affac0297b71bebada5)

2 years agoclient: only wait for write MDS OPs when unmounting 52304/head
Xiubo Li [Thu, 4 May 2023 06:22:37 +0000 (14:22 +0800)]
client: only wait for write MDS OPs when unmounting

We do not care about the read MDS OPs and it's safe by just dropping
them when unmounting.

Fixes: https://tracker.ceph.com/issues/59705
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit ee8443d63a175b9e5a3c60ba4e06270ae688d910)

2 years agoclient: flush the mdlog before waiting for unsafe requests
Xiubo Li [Thu, 4 May 2023 05:57:11 +0000 (13:57 +0800)]
client: flush the mdlog before waiting for unsafe requests

For safe replies the clients need to wait the MDS daemons to flush
the journal logs to Rados first. The worst case it will take 5 seconds.

Just trigger to flush the journal logs before waiting.

Fixes: https://tracker.ceph.com/issues/59705
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 2b715b858e36a82b42b12129fcfcc13090f30a93)