]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoMerge pull request #45464 from cfsnyder/wip-53471-pacific
Yuri Weinstein [Mon, 28 Mar 2022 14:19:13 +0000 (07:19 -0700)]
Merge pull request #45464 from cfsnyder/wip-53471-pacific

pacific: common: avoid pthread_mutex_unlock twice

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45436 from cfsnyder/wip-51783-pacific
Yuri Weinstein [Mon, 28 Mar 2022 14:17:47 +0000 (07:17 -0700)]
Merge pull request #45436 from cfsnyder/wip-51783-pacific

pacific: qa/rgw: add failing tempest test to blocklist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45163 from dvanders/wip-52635-pacific
Yuri Weinstein [Mon, 28 Mar 2022 14:16:18 +0000 (07:16 -0700)]
Merge pull request #45163 from dvanders/wip-52635-pacific

pacific: mds: ensure that we send the btime in cap messages

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #45565 from ajarr/wip-52875-pacific
Yuri Weinstein [Mon, 28 Mar 2022 14:15:32 +0000 (07:15 -0700)]
Merge pull request #45565 from ajarr/wip-52875-pacific

pacific: qa: fsync dir for asynchronous creat on stray tests

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #45564 from ajarr/wip-52427-pacific
Yuri Weinstein [Mon, 28 Mar 2022 14:13:50 +0000 (07:13 -0700)]
Merge pull request #45564 from ajarr/wip-52427-pacific

pacific: qa: ignore expected metadata cluster log error

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #44771 from pdvian/wip-53944-pacific
Yuri Weinstein [Fri, 25 Mar 2022 19:01:00 +0000 (12:01 -0700)]
Merge pull request #44771 from pdvian/wip-53944-pacific

pacific: osd/OSD: Log aggregated slow ops detail to cluster logs

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44664 from gregsfortytwo/wip-53933-pacific
Yuri Weinstein [Fri, 25 Mar 2022 18:59:57 +0000 (11:59 -0700)]
Merge pull request #44664 from gregsfortytwo/wip-53933-pacific

pacific: osd: PeeringState: fix selection order in calc_replicated_acting_stretch

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45491 from cfsnyder/wip-54077-pacific
David Galloway [Fri, 25 Mar 2022 18:48:44 +0000 (14:48 -0400)]
Merge pull request #45491 from cfsnyder/wip-54077-pacific

pacific: rgw: bucket chown bad memory usage

3 years agoMerge pull request #45557 from vshankar/wip-53911
David Galloway [Fri, 25 Mar 2022 18:45:54 +0000 (14:45 -0400)]
Merge pull request #45557 from vshankar/wip-53911

pacific: Revert "mds: kill session when mds do ms_handle_remote_reset"

3 years agoMerge pull request #45489 from cfsnyder/wip-54041-pacific
David Galloway [Fri, 25 Mar 2022 16:50:15 +0000 (12:50 -0400)]
Merge pull request #45489 from cfsnyder/wip-54041-pacific

pacific: rgw: remove bucket API returns NoSuchKey than NoSuchBucket

3 years agoMerge pull request #45543 from ajarr/rraja-pacific-backport-pr-44795
David Galloway [Fri, 25 Mar 2022 16:49:33 +0000 (12:49 -0400)]
Merge pull request #45543 from ajarr/rraja-pacific-backport-pr-44795

pacific: mgr/nfs: allow dynamic update of cephfs nfs export

3 years agoMerge pull request #45502 from cfsnyder/wip-54148-pacific
David Galloway [Fri, 25 Mar 2022 16:48:12 +0000 (12:48 -0400)]
Merge pull request #45502 from cfsnyder/wip-54148-pacific

pacific: rgw: RGWPostObj::execute() may lost data.

3 years agoMerge pull request #45500 from cfsnyder/wip-54147-pacific
David Galloway [Fri, 25 Mar 2022 16:47:35 +0000 (12:47 -0400)]
Merge pull request #45500 from cfsnyder/wip-54147-pacific

pacific: rgw/admin: fix radosgw-admin datalog list max-entries issue

3 years agoMerge pull request #45497 from cfsnyder/wip-54092-pacific
David Galloway [Fri, 25 Mar 2022 16:46:16 +0000 (12:46 -0400)]
Merge pull request #45497 from cfsnyder/wip-54092-pacific

pacific: rgwlc:  warn on missing RGW_ATTR_LC

3 years agoMerge pull request #45495 from cfsnyder/wip-54083-pacific
David Galloway [Fri, 25 Mar 2022 16:45:58 +0000 (12:45 -0400)]
Merge pull request #45495 from cfsnyder/wip-54083-pacific

pacific: librgw: make rgw file handle versioned

3 years agoMerge pull request #45487 from mchangir/wip-54532-pacific
David Galloway [Fri, 25 Mar 2022 16:44:12 +0000 (12:44 -0400)]
Merge pull request #45487 from mchangir/wip-54532-pacific

pacific: mds,client: add new getvxattr op

3 years agoMerge pull request #45555 from idryomov/wip-rbd-du-validate-pacific
Yuri Weinstein [Fri, 25 Mar 2022 14:48:41 +0000 (07:48 -0700)]
Merge pull request #45555 from idryomov/wip-rbd-du-validate-pacific

pacific: test/librbd: add test to verify diff_iterate size

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #44551 from lxbsz/wip-53759
Yuri Weinstein [Fri, 25 Mar 2022 14:40:14 +0000 (07:40 -0700)]
Merge pull request #44551 from lxbsz/wip-53759

pacific: mds: reset heartbeat in each MDSContext complete()

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #45509 from cfsnyder/wip-54152-pacific
Yuri Weinstein [Thu, 24 Mar 2022 23:30:07 +0000 (16:30 -0700)]
Merge pull request #45509 from cfsnyder/wip-54152-pacific

pacific: rgw: in bucket reshard list, clarify new num shards is tentative

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45508 from phlogistonjohn/jjm-backport-pacific-pr-44845-44870...
Adam King [Thu, 24 Mar 2022 20:51:35 +0000 (16:51 -0400)]
Merge pull request #45508 from phlogistonjohn/jjm-backport-pacific-pr-44845-44870-45211

pacific: mgr/nfs: support managing exports without orchestration enabled

Reviewed-by: Ramana Raja <rraja@redhat.com>
3 years agoMerge pull request #45569 from mgfritch/backport-45420-pacific
Adam King [Thu, 24 Mar 2022 20:50:09 +0000 (16:50 -0400)]
Merge pull request #45569 from mgfritch/backport-45420-pacific

pacific: cephadm: infer the default container image during pull

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45582 from idryomov/wip-rbd-nbd-cookie-pacific
Yuri Weinstein [Thu, 24 Mar 2022 20:26:08 +0000 (13:26 -0700)]
Merge pull request #45582 from idryomov/wip-rbd-nbd-cookie-pacific

pacific: backport nbd cookie support

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #45580 from idryomov/wip-fix-pids-limit-pacific
Yuri Weinstein [Thu, 24 Mar 2022 20:25:25 +0000 (13:25 -0700)]
Merge pull request #45580 from idryomov/wip-fix-pids-limit-pacific

pacific: cephadm: Remove containers pids-limit

Reviewed-by: Adam King adking@redhat.com
3 years agoMerge pull request #45477 from cfsnyder/wip-53637-pacific
Yuri Weinstein [Thu, 24 Mar 2022 20:21:45 +0000 (13:21 -0700)]
Merge pull request #45477 from cfsnyder/wip-53637-pacific

pacific: test/rgw: disable cls_rgw_gc test cases with defer_gc()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45476 from cfsnyder/wip-53518-pacific
Yuri Weinstein [Thu, 24 Mar 2022 20:21:04 +0000 (13:21 -0700)]
Merge pull request #45476 from cfsnyder/wip-53518-pacific

pacific: rgw: add object null point judging when listing pubsub  topics

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45484 from cfsnyder/wip-53835-pacific
Yuri Weinstein [Thu, 24 Mar 2022 14:55:41 +0000 (07:55 -0700)]
Merge pull request #45484 from cfsnyder/wip-53835-pacific

pacific: rgw: document S3 bucket replication support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45483 from cfsnyder/wip-53818-pacific
Yuri Weinstein [Thu, 24 Mar 2022 14:55:05 +0000 (07:55 -0700)]
Merge pull request #45483 from cfsnyder/wip-53818-pacific

pacific: rgw: RadosBucket::get_bucket_info() updates RGWBucketEnt

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45480 from cfsnyder/wip-53654-pacific
Yuri Weinstein [Thu, 24 Mar 2022 14:54:28 +0000 (07:54 -0700)]
Merge pull request #45480 from cfsnyder/wip-53654-pacific

pacific: rgw: init bucket index only if putting bucket instance info succeeds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45478 from cfsnyder/wip-53639-pacific
Yuri Weinstein [Thu, 24 Mar 2022 14:52:23 +0000 (07:52 -0700)]
Merge pull request #45478 from cfsnyder/wip-53639-pacific

pacific: rgw/amqp: add default case to silence compiler warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45463 from cfsnyder/wip-53254-pacific
Yuri Weinstein [Thu, 24 Mar 2022 14:38:43 +0000 (07:38 -0700)]
Merge pull request #45463 from cfsnyder/wip-53254-pacific

pacific: rgw: fix rgw.none statistics

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45596 from neha-ojha/wip-45512-pacific
Ilya Dryomov [Wed, 23 Mar 2022 20:52:49 +0000 (21:52 +0100)]
Merge pull request #45596 from neha-ojha/wip-45512-pacific

pacific: ceph/admin: s/master/main

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #45532 from idryomov/wip-fix-schedule-status-pacific
Yuri Weinstein [Wed, 23 Mar 2022 20:28:57 +0000 (13:28 -0700)]
Merge pull request #45532 from idryomov/wip-fix-schedule-status-pacific

pacific: mgr/rbd_support: cast pool_id from int to str when collecting LevelSpec

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
3 years agorgw: fix rgw.none statistics 45463/head
J. Eric Ivancich [Wed, 8 Sep 2021 18:58:02 +0000 (14:58 -0400)]
rgw: fix rgw.none statistics

Only unaccount for objects being removed if they exist.

Also, clean up some formatting.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 95b662fa176cd841e26b76341f91bf2e6c266003)

3 years agoceph/admin: s/master/main 45596/head
Zac Dover [Thu, 17 Mar 2022 23:05:45 +0000 (09:05 +1000)]
ceph/admin: s/master/main

This PR changes the name "master" to "main" so
that builds (and, I assume, a great many other
things) will not fail.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 6a1dd3a8a2f3dc9fe8615d402c9041273516ff89)

3 years agoqa/workunits/rbd: test map/attach with --show-cookie/--cookie options 45582/head
Prasanna Kumar Kalever [Mon, 17 May 2021 08:47:53 +0000 (14:17 +0530)]
qa/workunits/rbd: test map/attach with --show-cookie/--cookie options

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 2adfe55bcce08f8f46989e9ddbf46aa74e681992)

3 years agorbd-nbd: allow user to specify cookie at map
Prasanna Kumar Kalever [Wed, 11 Aug 2021 10:02:25 +0000 (15:32 +0530)]
rbd-nbd: allow user to specify cookie at map

Allow user to specify cookie of choice at the time of map

$ rbd device attach rbd-pool/image --device /dev/nbd0 \
--cookie 6f85d970-10b2-456b-8baf-676aa4d782e4 --options try-netlink

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 16404dede832f3908d48fc36c7714891bfcfde20)

3 years agorbd: add --show-cookie/--cookie option for map/attach commands
Prasanna Kumar Kalever [Fri, 18 Jun 2021 06:48:35 +0000 (12:18 +0530)]
rbd: add --show-cookie/--cookie option for map/attach commands

Example:
$ rbd device map rbd-pool/image --show-cookie --try-netlink --device-type nbd

$ rbd device attach rbd-pool/image --device /dev/nbd0 \
      --cookie 6f85d970-10b2-456b-8baf-676aa4d782e4 --device-type nbd

older Kernel versions can use --force to skip the cookie validation

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6df06c7a62bdd6e9ce61b26c738dcd1509e68273)

Conflicts:
src/tools/rbd/action/Device.cc [ commit e1b4811bc324 ("rbd: add
  missing switch arguments for recognition by get_command_spec()")
  already in pacific -- re-add show-cookie ]

3 years agorbd-nbd: allow attach without --cookie for old kernel versions
Prasanna Kumar Kalever [Thu, 17 Jun 2021 13:35:20 +0000 (19:05 +0530)]
rbd-nbd: allow attach without --cookie for old kernel versions

For backward compatibility allow attach without --cookie option:

[root@linux-vm1]# rbd-nbd attach rbd-pool/image0 --device /dev/nbd0
/dev/nbd0

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 833e71282ad82023237c5eb407b9fa4630280c15)

3 years agorbd-nbd: show per device cookie with list command
Prasanna Kumar Kalever [Thu, 13 May 2021 07:09:25 +0000 (12:39 +0530)]
rbd-nbd: show per device cookie with list command

[root@linux-vm1]# rbd-nbd list-mapped
id    pool      namespace  image   snap  device     cookie
8133  rbd-pool             image0  -     /dev/nbd0  c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit efeb255d107075516c95f719f9a2ec9375d5bb2f)

3 years agorbd-nbd: mandate device cookie for attach command
Prasanna Kumar Kalever [Thu, 13 May 2021 08:22:25 +0000 (13:52 +0530)]
rbd-nbd: mandate device cookie for attach command

[root@linux-vm1]# rbd-nbd attach rbd-pool/image0 --device /dev/nbd0 \
                          --cookie c704cb91-c6cf-466e-a335-0e935c0d5e47
/dev/nbd0

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6ca70ee4f078aba86a6edd013bd8d95f690a46a1)

3 years agorbd-nbd: provide a flag to show device cookie with map command
Prasanna Kumar Kalever [Wed, 12 May 2021 13:03:54 +0000 (18:33 +0530)]
rbd-nbd: provide a flag to show device cookie with map command

[root@linux-vm1]# rbd-nbd map rbd-pool/image0 --try-netlink --show-cookie
/dev/nbd0 c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 59c08eabf12fdcf155b54dc422fb0f67d6ad747e)

3 years agorbd-nbd: generate and send device cookie with netlink connect request
Prasanna Kumar Kalever [Wed, 12 May 2021 13:00:46 +0000 (18:30 +0530)]
rbd-nbd: generate and send device cookie with netlink connect request

[root@linux-vm1]# rbd-nbd map rbd-pool/image0 --try-netlink
/dev/nbd0

[root@linux-vm1]# cat /sys/block/nbd0/backend
c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit eee8d8a32e3f39440ac417b2e9ea46d93d4a1d73)

3 years agoqa/workunits/rbd: use rbd cli for device attach/detach commands
Prasanna Kumar Kalever [Tue, 25 May 2021 12:24:29 +0000 (17:54 +0530)]
qa/workunits/rbd: use rbd cli for device attach/detach commands

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit e2aa4128f1e451a85200441f70a13336e843dad6)

3 years agorbd: improve conditional compilation specific checks
Prasanna Kumar Kalever [Mon, 17 May 2021 09:40:45 +0000 (15:10 +0530)]
rbd: improve conditional compilation specific checks

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 4d7db9e86a246bf77e8545d0d4e220d90f48ac29)

3 years agorbd: promote rbd-nbd attach and detach at rbd integrated cli
Prasanna Kumar Kalever [Thu, 6 May 2021 07:27:56 +0000 (12:57 +0530)]
rbd: promote rbd-nbd attach and detach at rbd integrated cli

Example:
$ rbd device attach rbd-pool/image --device /dev/nbd0 --device-type nbd --force
$ rbd device detach rpool/image --device-type nbd

for now returning EOPNOTSUPP with krbd, ggate and wnbd

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 35e5352f719857bed99caa4c76ae19bb68918c8e)

Conflicts:
src/tools/rbd/action/Device.cc [ commit e1b4811bc324 ("rbd: add
  missing switch arguments for recognition by get_command_spec()")
  already in pacific ]
src/tools/rbd/action/Wnbd.cc [ ditto ]

3 years agoqa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS 45580/head
Ilya Dryomov [Tue, 22 Mar 2022 10:36:18 +0000 (11:36 +0100)]
qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f0ade57458b93f8401de8670ae62bf2295a6c40c)

[ commit 1f714da81440 ("qa: fix or add missing .qa links") not in
  pacific -- added qa/suites/orch/cephadm/workunits/task/.qa ]

3 years agocephadm: remove containers pids-limit
Teoman ONAY [Thu, 11 Nov 2021 15:05:49 +0000 (15:05 +0000)]
cephadm: remove containers pids-limit

The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>
(cherry picked from commit de8b3c2676e65eb61df54c65cfd3b3af1e68da56)

3 years agoMerge pull request #45426 from cfsnyder/wip-51348-pacific
Yuri Weinstein [Wed, 23 Mar 2022 01:05:10 +0000 (18:05 -0700)]
Merge pull request #45426 from cfsnyder/wip-51348-pacific

pacific: rgw/notification: send correct size in COPY events

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45236 from k0ste/wip-54451-pacific
Yuri Weinstein [Wed, 23 Mar 2022 01:03:42 +0000 (18:03 -0700)]
Merge pull request #45236 from k0ste/wip-54451-pacific

pacific: mgr/prometheus: Added `avail_raw` field for Pools DF Prometheus mgr module

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45397 from kamoltat/wip-ksirivad-pacific-backport-45078
Yuri Weinstein [Tue, 22 Mar 2022 23:43:54 +0000 (16:43 -0700)]
Merge pull request #45397 from kamoltat/wip-ksirivad-pacific-backport-45078

pacific: mon/MonCommands.h: fix target_size_ratio range

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45376 from ronen-fr/wip-rf-43303-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:43:36 +0000 (16:43 -0700)]
Merge pull request #45376 from ronen-fr/wip-rf-43303-pacific

pacific: osd: fix the truncation of an int by int division

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45373 from smithfarm/wip-54276-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:43:04 +0000 (16:43 -0700)]
Merge pull request #45373 from smithfarm/wip-54276-pacific

pacific: common: fix FTBFS due to dout & need_dynamic on GCC-12

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #45323 from ljflores/wip-54466-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:41:57 +0000 (16:41 -0700)]
Merge pull request #45323 from ljflores/wip-54466-pacific

pacific: osd: require osd_pg_max_concurrent_snap_trims > 0

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
3 years agoMerge pull request #45319 from ljflores/wip-52078-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:40:56 +0000 (16:40 -0700)]
Merge pull request #45319 from ljflores/wip-52078-pacific

pacific: test: fix wrong alarm (HitSetWrite)

Reviewed-by: Myoungwon Oh <omwmw@sk.com>
3 years agoMerge pull request #45152 from chrisphoffman/wip-53644-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:38:12 +0000 (16:38 -0700)]
Merge pull request #45152 from chrisphoffman/wip-53644-pacific

pacific: mon: Omit MANY_OBJECTS_PER_PG warning when autoscaler is on

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
3 years agoMerge pull request #43698 from cfsnyder/wip-52636-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:36:46 +0000 (16:36 -0700)]
Merge pull request #43698 from cfsnyder/wip-52636-pacific

pacific: mon: do not quickly mark mds laggy when MON_DOWN

Reviewed-by: Venky Shankar vshankar@redhat.com
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #45459 from cfsnyder/wip-53079-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:30:17 +0000 (16:30 -0700)]
Merge pull request #45459 from cfsnyder/wip-53079-pacific

pacific: src/rgw: Fix for malformed url

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45457 from cfsnyder/wip-53036-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:29:48 +0000 (16:29 -0700)]
Merge pull request #45457 from cfsnyder/wip-53036-pacific

pacific: rgw: cls_bucket_list_unordered() might return one redundent entry every time is_truncated is true

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45442 from cfsnyder/wip-52107-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:29:23 +0000 (16:29 -0700)]
Merge pull request #45442 from cfsnyder/wip-52107-pacific

pacific: radosgw-admin: 'sync status' is not behind if there are no mdlog entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45440 from cfsnyder/wip-52071-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:28:54 +0000 (16:28 -0700)]
Merge pull request #45440 from cfsnyder/wip-52071-pacific

pacific: rgw: add the condition of lock mode conversion to PutObjRentention

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45427 from cfsnyder/wip-51351-pacific
Yuri Weinstein [Tue, 22 Mar 2022 23:28:19 +0000 (16:28 -0700)]
Merge pull request #45427 from cfsnyder/wip-51351-pacific

pacific: rgw/amqp: remove the explicit "disconnect()" interface

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45445 from cfsnyder/wip-52115-pacific
Yuri Weinstein [Tue, 22 Mar 2022 21:58:04 +0000 (14:58 -0700)]
Merge pull request #45445 from cfsnyder/wip-52115-pacific

pacific: qa/rgw: update apache-maven mirror for rgw/hadoop-s3a

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45439 from cfsnyder/wip-52050-pacific
Yuri Weinstein [Tue, 22 Mar 2022 21:57:25 +0000 (14:57 -0700)]
Merge pull request #45439 from cfsnyder/wip-52050-pacific

pacific: rgw: default ms_mon_client_mode = secure

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45434 from cfsnyder/wip-51777-pacific
Yuri Weinstein [Tue, 22 Mar 2022 21:56:47 +0000 (14:56 -0700)]
Merge pull request #45434 from cfsnyder/wip-51777-pacific

pacific: rgw: forward request in multisite for RGWDeleteBucketPolicy and RGWDeleteBucketPublicAccessBlock

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45430 from cfsnyder/wip-51701-pacific
Yuri Weinstein [Tue, 22 Mar 2022 21:56:08 +0000 (14:56 -0700)]
Merge pull request #45430 from cfsnyder/wip-51701-pacific

pacific: rgw: url_decode before parsing copysource in copyobject

3 years agoMerge pull request #43956 from cfsnyder/wip-52782-pacific
Yuri Weinstein [Tue, 22 Mar 2022 21:55:42 +0000 (14:55 -0700)]
Merge pull request #43956 from cfsnyder/wip-52782-pacific

pacific: rgw/sts: adding role name and role session to ops log.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agocephadm: infer the default container image during pull 45569/head
Michael Fritch [Wed, 16 Mar 2022 13:39:03 +0000 (07:39 -0600)]
cephadm: infer the default container image during pull

Fixes: https://tracker.ceph.com/issues/54588
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 736af2b9c29e375a184b863220641b7d254e7d04)

3 years agoMerge pull request #45455 from cfsnyder/wip-52990-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:48:35 +0000 (13:48 -0700)]
Merge pull request #45455 from cfsnyder/wip-52990-pacific

pacific: rgw: have "bucket check --fix" fix pool ids correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45461 from cfsnyder/wip-53158-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:47:39 +0000 (13:47 -0700)]
Merge pull request #45461 from cfsnyder/wip-53158-pacific

pacific: rgw:When KMS encryption is used and the key does not exist, we should…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45453 from cfsnyder/wip-52988-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:46:57 +0000 (13:46 -0700)]
Merge pull request #45453 from cfsnyder/wip-52988-pacific

pacific: rgw: document rgw_lc_debug_interval configuration option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45450 from cfsnyder/wip-52611-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:46:08 +0000 (13:46 -0700)]
Merge pull request #45450 from cfsnyder/wip-52611-pacific

pacific: rgw: clean-up logging of function entering to make thorough and consistent

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45448 from cfsnyder/wip-52348-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:45:27 +0000 (13:45 -0700)]
Merge pull request #45448 from cfsnyder/wip-52348-pacific

pacific: rgw: change order of xml elements in ListRoles response

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45444 from cfsnyder/wip-52113-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:44:33 +0000 (13:44 -0700)]
Merge pull request #45444 from cfsnyder/wip-52113-pacific

pacific: qa/rgw: barbican and pykmip tasks upgrade pip before installing pytz

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45438 from cfsnyder/wip-51810-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:43:46 +0000 (13:43 -0700)]
Merge pull request #45438 from cfsnyder/wip-51810-pacific

pacific: RGW - Zipper - Make default args match in get_obj_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45432 from cfsnyder/wip-51703-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:43:20 +0000 (13:43 -0700)]
Merge pull request #45432 from cfsnyder/wip-51703-pacific

pacific: rgw: fix md5 not match for RGWBulkUploadOp upload when enable rgw com…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45428 from cfsnyder/wip-51352-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:42:45 +0000 (13:42 -0700)]
Merge pull request #45428 from cfsnyder/wip-51352-pacific

pacific: rgw: write meta of a MP part to a correct pool

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #45421 from cfsnyder/wip-54426-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:41:56 +0000 (13:41 -0700)]
Merge pull request #45421 from cfsnyder/wip-54426-pacific

pacific: rgw: add OPT_BUCKET_SYNC_RUN to gc_ops_list, so that

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45413 from cfsnyder/wip-54035-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:41:23 +0000 (13:41 -0700)]
Merge pull request #45413 from cfsnyder/wip-54035-pacific

pacific: qa: move certificates for kmip task into /etc/ceph

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #45411 from cfsnyder/wip-54494-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:40:53 +0000 (13:40 -0700)]
Merge pull request #45411 from cfsnyder/wip-54494-pacific

pacific: rgw: fix segfault in UserAsyncRefreshHandler::init_fetch

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44729 from cfsnyder/wip-53849-pacific
Yuri Weinstein [Tue, 22 Mar 2022 20:40:05 +0000 (13:40 -0700)]
Merge pull request #44729 from cfsnyder/wip-53849-pacific

pacific: rgwlc:  remove lc entry on bucket delete

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoqa: fsync dir for asynchronous creat on stray tests 45565/head
Patrick Donnelly [Tue, 14 Sep 2021 14:58:20 +0000 (10:58 -0400)]
qa: fsync dir for asynchronous creat on stray tests

Use the enhanced create_n_files to dedup code. Also split the large test
into three.

Fixes: https://tracker.ceph.com/issues/52606
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d33debc6436732906dfdd908bafeb8d956ff589e)

3 years agoqa: refactor and generalize create_n_files
Patrick Donnelly [Tue, 14 Sep 2021 17:45:33 +0000 (13:45 -0400)]
qa: refactor and generalize create_n_files

Few things:

- Allow calling fsync on directory (to support async create kernel).
- Allow immediately unlinking the created file (for stray testing).
- Close any file descriptors created.
- Write unique content (the i variable) to each file.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 395d20a2b7cfd081c1f1a172da14d183e3fe9003)

3 years agomds: improve debugging for fragment size check
Patrick Donnelly [Wed, 6 Oct 2021 15:09:48 +0000 (11:09 -0400)]
mds: improve debugging for fragment size check

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5ae32803e5a9a933953082b5054407a128a00bd7)

3 years agomon/FSCommands: add 'recover' flag in `fs new` command
Ramana Raja [Wed, 11 Aug 2021 20:34:47 +0000 (16:34 -0400)]
mon/FSCommands: add 'recover' flag in `fs new` command

Currently, to recover a file system after recovering monitor store, you
need to stop all the MDSs; create FSMap with defaults using `fs new`
command; execute `fs reset` command to get the file system's rank 0 into
existing but failed state; and then restart MDSs.

Add 'recover' flag to the `fs new` command that sets the file system's
rank 0 to existing but failed state, and sets the file system's
'joinable' setting to False. Using the `fs new` command with 'recover'
flag gets rid of the steps to stop all the MDSs and execute `fs reset`
command when recovering the file system after recoving monitor store.

Fixes: https://tracker.ceph.com/issues/51716
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 67bb13859a248a03059507ec52b4f551db0a72c9)

3 years agodoc/cephfs: recover file system after recovering
Ramana Raja [Tue, 13 Jul 2021 05:10:50 +0000 (01:10 -0400)]
doc/cephfs: recover file system after recovering

... monitor stores using OSDs. The steps are valid only to recover
single active MDS file systems.

Partially-fixes: https://tracker.ceph.com/issues/51341
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit ffe5cfb6875d038fa5a44715bf6ac87cd7184471)

3 years agoMerge pull request #44885 from BenoitKnecht/wip-54118-pacific
Ernesto Puerta [Tue, 22 Mar 2022 18:53:19 +0000 (19:53 +0100)]
Merge pull request #44885 from BenoitKnecht/wip-54118-pacific

pacific: mgr: Fix ceph_daemon label in ceph_rgw_* metrics

Reviewed-by: BenoitKnecht <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoqa: ignore expected metadata cluster log error 45564/head
Patrick Donnelly [Wed, 28 Jul 2021 16:17:19 +0000 (09:17 -0700)]
qa: ignore expected metadata cluster log error

Fixes: https://tracker.ceph.com/issues/51905
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8116b95a345db28a9b5d82ff961f79b1bbc4e9f7)

3 years agoMerge pull request #45351 from nmshelke/wip-54478-pacific
Yuri Weinstein [Tue, 22 Mar 2022 14:18:21 +0000 (07:18 -0700)]
Merge pull request #45351 from nmshelke/wip-54478-pacific

pacific: ceph-fuse: perform cleanup if test_dentry_handling failed

Reviewed-by: Venky Shankar vshankar@redhat.com
Reviewed-by: Kotresh HR khiremat@redhat.com
3 years agoMerge pull request #45227 from rkachach/backport_of_pr_44928
Yuri Weinstein [Tue, 22 Mar 2022 14:16:37 +0000 (07:16 -0700)]
Merge pull request #45227 from rkachach/backport_of_pr_44928

pacific: mgr/cephadm: validating service_id for MDS

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Adam King adking@redhat.com
3 years agoMerge pull request #45205 from kotreshhr/pacific-mgr-volumes-backports-0
Yuri Weinstein [Tue, 22 Mar 2022 14:14:59 +0000 (07:14 -0700)]
Merge pull request #45205 from kotreshhr/pacific-mgr-volumes-backports-0

pacific: mgr/volumes: A few mgr volumes pacific backports

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoRevert "mds: kill session when mds do ms_handle_remote_reset" 45557/head
Venky Shankar [Tue, 22 Mar 2022 11:48:03 +0000 (17:18 +0530)]
Revert "mds: kill session when mds do ms_handle_remote_reset"

This reverts commit 749ee198ecfffac67a896448229edce2e19ba633.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agotest/librbd: add test to verify diff_iterate size 45555/head
Christopher Hoffman [Mon, 7 Mar 2022 18:35:56 +0000 (18:35 +0000)]
test/librbd: add test to verify diff_iterate size

Add test case to verify diff size values of image and multiple
snapshots.

Fixes: https://tracker.ceph.com/issues/54440
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
(cherry picked from commit d4e44df1be2bafa1c0ceabc73bb7243104fc7ad4)

3 years agocephfs/test_nfs: Add test for dynamic update export 45543/head
Ramana Raja [Fri, 11 Mar 2022 19:49:42 +0000 (14:49 -0500)]
cephfs/test_nfs: Add test for dynamic update export

Add test to verify that the NFS servers don't restart when the
access type of a CephFS NFS export is updated.

And check the NFS servers are restarted when the pseudo path of
a CephFS NFS export is updated.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit d7904e78d13c24b305b4e83f15ee2b86ab2085e5)

3 years agocephfs/test_nfs: fix _write_to_read_only_export()
Ramana Raja [Fri, 11 Mar 2022 21:36:44 +0000 (16:36 -0500)]
cephfs/test_nfs: fix _write_to_read_only_export()

... to fail the test if write to read only export succeeded.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 3c22f6eb716b4b4f2817566a5b14350f7dd3310e)

3 years agomgr/nfs: Don't notify ganesha rados object
Ramana Raja [Wed, 26 Jan 2022 04:30:26 +0000 (23:30 -0500)]
mgr/nfs: Don't notify ganesha rados object

... to update export if the ganesha server is going to be
restarted. The ganesha server restart will reload the exports.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 4df294b7fafc308611b75c4656a40c42d1d95361)

3 years agomgr/nfs: allow dynamic update of cephfs nfs export
Ramana Raja [Tue, 25 Jan 2022 01:06:11 +0000 (20:06 -0500)]
mgr/nfs: allow dynamic update of cephfs nfs export

mgr/nfs module's apply_export() method is used to update an existing
CephFS NFS export. The method always restarted the ganesha service (
ganesha server cluster) after updating the export object and notifying
the ganesha servers to reload their exports. The restart temporarily
affected the clients connections of all the exports served by the
ganesha servers.

It is not always necessary to restart the ganesha servers. Only
updating the export ID, path, or FSAL block of a CephFS NFS export
requires a restart. So modify apply_export() to only restart the
ganesha servers for such export updates.

The mgr/nfs module creates a FSAL ceph user with read-only or
read-write path restricted MDS caps for each export. To change the
access type of the CephFS NFS export, the MDS caps of the export's FSAL
ceph user must also be changed. Ganesha can dynamically enforce an
export's access type changes, but Ceph server daemons can't dynamically
enforce changes in caps of the Ceph clients. To allow dynamic updates
of CephFS NFS exports, always create a FSAL Ceph user with read-write
path restricted MDS caps per export. Rely on the ganesha servers to
enforce the export access type changes for the NFS clients.

Fixes: https://tracker.ceph.com/issues/54025
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 2415e03217b4afe9e430863da0b4503be254c425)

3 years agoqa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert 45532/head
Ilya Dryomov [Sat, 19 Mar 2022 13:04:52 +0000 (14:04 +0100)]
qa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert

Commit 08df6e0fd006 ("qa/workunits/rbd: expand LevelSpec parsing
coverage") didn't account for images with a separate data pool.  This
was missed because of small-cache-pool.yaml breakage.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 90a90ad47dd3140c796ef4da7263c9633d34e841)

3 years agorbd: fix a typo in "rbd trash purge schedule ls" formatted output
Ilya Dryomov [Sat, 5 Mar 2022 08:59:36 +0000 (09:59 +0100)]
rbd: fix a typo in "rbd trash purge schedule ls" formatted output

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit beffb7f9c6a4a19b6c4cfb65d04386575af50961)

3 years agoqa/workunits/rbd: expand LevelSpec parsing coverage
Ilya Dryomov [Fri, 4 Mar 2022 13:52:43 +0000 (14:52 +0100)]
qa/workunits/rbd: expand LevelSpec parsing coverage

Invoke "rbd mirror snapshot schedule ls -R" and "rbd mirror snapshot
schedule status" commands on all levels, consistently.  In particular,
make sure that an image level schedule is listed for a recursive query
at the pool level both before and after the schedule kicks in:

  $ rbd create --size 1G --mirror-image-mode snapshot -p foo bar
  $ rbd mirror snapshot schedule add -p foo --image bar 1m
  $ rbd mirror snapshot schedule ls -p foo -R
  POOL  NAMESPACE  IMAGE  SCHEDULE
  foo              bar    every 1m
  <wait for schedule to become visible in status>
  $ rbd mirror snapshot schedule ls -p foo -R
  POOL  NAMESPACE  IMAGE  SCHEDULE
  foo              bar    every 1m

Also, make sure that pool and image level status queries work:

  $ rbd mirror snapshot schedule status -p foo
  SCHEDULE TIME        IMAGE
  2022-03-04 07:14:00  foo/bar
  $ rbd mirror snapshot schedule status -p foo --image bar
  SCHEDULE TIME        IMAGE
  2022-03-04 07:14:00  foo/bar

Both of these issues are fixed by the previous commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 08df6e0fd00689649b802e975df320efe3b07c83)

3 years agorgw: in bucket reshard list, clarify new num shards is tentative 45509/head
J. Eric Ivancich [Wed, 22 Dec 2021 19:45:59 +0000 (14:45 -0500)]
rgw: in bucket reshard list, clarify new num shards is tentative

With dynamic bucket index resharding, when the average number of
objects per shard exceeds the configured value, that bucket is
scheduled for reshard. That bucket may receive more new objects before
the resharding takes place. As a result, the existing code
re-calculates the number of new shards just prior to resharding,
rather than waste a resharding opportunity with too low a value.

The same holds true for a user-scheduled resharding.

A user reported confusion that the number reported in `radosgw-admin
reshard list` wasn't the number that the reshard operation ultimately
used. This commit makes it clear that the new number of shards is
"tentative". And test_rgw_reshard.py is updated to reflect this
altered output.

Additionally this commit adds some modernization and efficiency to the
"reshard list" subcommand.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit aa0071ce8b8594b92c0bed2be7a9bf35bfff8cac)

Conflicts:
src/rgw/rgw_admin.cc

Cherry-pick notes:
- Confict caused by reshard.list not having dout prefix provider arg in Pacific