]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agodoc: document mds locking operation 44308/head
Venky Shankar [Tue, 14 Dec 2021 12:47:02 +0000 (18:17 +0530)]
doc: document mds locking operation

Signed-off-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44849 from rhcs-dashboard/bugfix-cephfs-invalid-dir-permission
Sarthak0702 [Fri, 11 Feb 2022 09:59:20 +0000 (15:29 +0530)]
Merge pull request #44849 from rhcs-dashboard/bugfix-cephfs-invalid-dir-permission

mgr/dashboard: Directories Menu Can't Use on Ceph File System Dashboard

3 years agoMerge pull request #44970 from kotreshhr/fix-clone-uid-gid
Venky Shankar [Fri, 11 Feb 2022 07:41:24 +0000 (13:11 +0530)]
Merge pull request #44970 from kotreshhr/fix-clone-uid-gid

mgr/volumes: Fix clone uid/gid mismatch

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44968 from cyx1231st/wip-crimson-cleanup-cleaner-1
Samuel Just [Thu, 10 Feb 2022 22:45:59 +0000 (14:45 -0800)]
Merge pull request #44968 from cyx1231st/wip-crimson-cleanup-cleaner-1

crimson/os/seastore/segment_cleaner: minor cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44964 from yaarith/pending-release-notes-perf
Neha Ojha [Thu, 10 Feb 2022 22:40:38 +0000 (14:40 -0800)]
Merge pull request #44964 from yaarith/pending-release-notes-perf

PendingReleaseNotes: add a note about perf channel

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44963 from ljflores/wip-peering-state-docs
Neha Ojha [Thu, 10 Feb 2022 22:07:31 +0000 (14:07 -0800)]
Merge pull request #44963 from ljflores/wip-peering-state-docs

doc/dev: add section on using the gen_state_diagram.py script

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agodoc/dev: add section on using the gen_state_diagram.py script 44963/head
Laura Flores [Wed, 9 Feb 2022 20:53:33 +0000 (20:53 +0000)]
doc/dev: add section on using the gen_state_diagram.py script

Commands sourced from Jianshen Liu's blog post on the Ceph peering state machine: https://jianshenliu.com/blog/development/ceph-peering-state-machine/

Also fixed the formatting of "PG info".

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agoMerge pull request #44765 from guits/guits-fix-cve-cv
Guillaume Abrioux [Thu, 10 Feb 2022 12:38:13 +0000 (13:38 +0100)]
Merge pull request #44765 from guits/guits-fix-cve-cv

ceph-volume: honour osd_dmcrypt_key_size option

3 years agoMerge pull request #44808 from rhcs-dashboard/support-snmp-service-creation
Ernesto Puerta [Thu, 10 Feb 2022 11:54:00 +0000 (12:54 +0100)]
Merge pull request #44808 from rhcs-dashboard/support-snmp-service-creation

mgr/dashboard: support snmp-gateway service creation from UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
3 years agomgr/volumes: Fix clone uid/gid mismatch 44970/head
Kotresh HR [Thu, 10 Feb 2022 05:34:41 +0000 (11:04 +0530)]
mgr/volumes: Fix clone uid/gid mismatch

This is the regression caused by commit 18b85c53a.
The 'set_attrs' function sets the uid/gid of the
group to the subvolume if uid/gid is not passed.
The attrs of the clone should match the source
snapshot. Hence, don't use the 'set_attrs'
function to set only the quota attrs for the
clone.

Fixes: https://tracker.ceph.com/issues/54066
Signed-off-by: Kotresh HR <khiremat@redhat.com>
3 years agoMerge pull request #44953 from aejv/spellingerror
Venky Shankar [Thu, 10 Feb 2022 06:51:47 +0000 (12:21 +0530)]
Merge pull request #44953 from aejv/spellingerror

purge_queue.py: fixed typo

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agocrimson/os/seastore/segment_cleaner: mark get_next_gc_target() private 44968/head
Yingxin Cheng [Thu, 10 Feb 2022 05:03:21 +0000 (13:03 +0800)]
crimson/os/seastore/segment_cleaner: mark get_next_gc_target() private

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/segment_cleaner: make init_mark_segment_closed() private
Yingxin Cheng [Thu, 10 Feb 2022 04:59:02 +0000 (12:59 +0800)]
crimson/os/seastore/segment_cleaner: make init_mark_segment_closed() private

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/segment_cleaner: drop unused code
Yingxin Cheng [Thu, 10 Feb 2022 04:57:36 +0000 (12:57 +0800)]
crimson/os/seastore/segment_cleaner: drop unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/net/socket: add a missing header
Yingxin Cheng [Thu, 10 Feb 2022 04:52:12 +0000 (12:52 +0800)]
crimson/net/socket: add a missing header

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoceph-volume/activate: load the config from lv tag 44765/head
Guillaume Abrioux [Thu, 10 Feb 2022 01:23:51 +0000 (02:23 +0100)]
ceph-volume/activate: load the config from lv tag

When `ceph-volume lvm trigger` is called with an OSD where the tag
`ceph.cluster_name` is not 'ceph', it fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume/tests: use centos/stream8 images
Guillaume Abrioux [Wed, 9 Feb 2022 17:33:27 +0000 (18:33 +0100)]
ceph-volume/tests: use centos/stream8 images

Since recent move from CentOS 8 to CentOS Stream 8, let's do the same here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume/tests: add tests in util/encryption.py
Guillaume Abrioux [Wed, 9 Feb 2022 16:04:19 +0000 (17:04 +0100)]
ceph-volume/tests: add tests in util/encryption.py

this adds some unit tests in order to cover `luks_format()` and `luks_open()`
in `util/encryption.py`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoceph-volume: honour osd_dmcrypt_key_size option
Guillaume Abrioux [Tue, 25 Jan 2022 09:25:53 +0000 (10:25 +0100)]
ceph-volume: honour osd_dmcrypt_key_size option

ceph-volume doesn't honour osd_dmcrypt_key_size.
It means the default size is always applied.

It also changes the default value in `get_key_size_from_conf()`

From cryptsetup manpage:

> For XTS mode you can optionally set a key size of 512 bits with the -s option.

Using more than 512bits will end up with the following error message:

```
Key size in XTS mode must be 256 or 512 bits.
```

Fixes: https://tracker.ceph.com/issues/54006
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoPendingReleaseNotes: add a note about perf channel 44964/head
Yaarit Hatuka [Wed, 9 Feb 2022 21:48:51 +0000 (21:48 +0000)]
PendingReleaseNotes: add a note about perf channel

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
3 years agoMerge pull request #44956 from rhcs-dashboard/appli-fix
Ernesto Puerta [Wed, 9 Feb 2022 19:44:55 +0000 (20:44 +0100)]
Merge pull request #44956 from rhcs-dashboard/appli-fix

mgr/dashboard: change the readFile to readFileSync

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44939 from soumyakoduri/wip-skoduri-dbstore-tests
Casey Bodley [Wed, 9 Feb 2022 18:01:22 +0000 (13:01 -0500)]
Merge pull request #44939 from soumyakoduri/wip-skoduri-dbstore-tests

rgw/dbstore: Add dbstore-tests to `make check`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agomgr/dashboard: change the readFile to readFileSync 44956/head
Nizamudeen A [Wed, 9 Feb 2022 15:36:16 +0000 (21:06 +0530)]
mgr/dashboard: change the readFile to readFileSync

Apparently the readFile i added in #44934 is async and that's not what
we want. so changing it to the synchronous call that is readFileSync

Fixes: https://tracker.ceph.com/issues/54190
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge PR #42000 into master
Patrick Donnelly [Wed, 9 Feb 2022 14:34:49 +0000 (09:34 -0500)]
Merge PR #42000 into master

* refs/pull/42000/head:
qa: update rhel kclient to setup container tools
qa: stop overriding distro for k-testing
qa: only use RHEL for workload testing
qa: convert fs:workload to use cephadm
qa: split fs begin task
qa/tasks/cephadm: setup CephManager when OSDs are provisioned
qa/tasks/cephadm: setup file system if MDS are provisioned

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agopurge_queue.py: fixed typo 44953/head
Judah van der Ster [Wed, 9 Feb 2022 14:25:58 +0000 (15:25 +0100)]
purge_queue.py: fixed typo

Signed-off-by: Judah van der Ster <jevanderster@gmail.com>
3 years agoMerge pull request #44937 from idryomov/wip-rbd-help-positional-optional
Ilya Dryomov [Wed, 9 Feb 2022 14:08:58 +0000 (15:08 +0100)]
Merge pull request #44937 from idryomov/wip-rbd-help-positional-optional

rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44492 from xijiacun/master
Casey Bodley [Wed, 9 Feb 2022 13:37:35 +0000 (08:37 -0500)]
Merge pull request #44492 from xijiacun/master

rgw: Erase old storage class attr when the object is rewrited using r…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agomgr/dashboard:Directories Menu Can't Use on Ceph File System Dashboard 44849/head
Sarthak0702 [Thu, 3 Feb 2022 17:59:17 +0000 (23:29 +0530)]
mgr/dashboard:Directories Menu Can't Use on Ceph File System Dashboard

Added exception handling to opendir() in cephfs.py for directories with no execute permission.

Fixes: https://tracker.ceph.com/issues/51611
Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
3 years agomgr/dashboard: add snmp-gateway service e2e tests 44808/head
Avan Thakkar [Tue, 1 Feb 2022 13:02:42 +0000 (18:32 +0530)]
mgr/dashboard: add snmp-gateway service e2e tests

Fixes: https://tracker.ceph.com/issues/54034
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
3 years agoMerge pull request #44934 from rhcs-dashboard/applitools-baseline-branch
Ernesto Puerta [Wed, 9 Feb 2022 11:05:21 +0000 (12:05 +0100)]
Merge pull request #44934 from rhcs-dashboard/applitools-baseline-branch

mgr/dashboard: set appropriate baseline branch for applitools

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
3 years agorgw/dbstore: Add dbstore-tests to `make check` 44939/head
Soumya Koduri [Tue, 8 Feb 2022 10:02:16 +0000 (15:32 +0530)]
rgw/dbstore: Add dbstore-tests to `make check`

Include and run dbstore-tests as part of `make check` target

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agomgr/dashboard: set appropriate baseline branch for applitools 44934/head
Nizamudeen A [Tue, 8 Feb 2022 06:20:29 +0000 (11:50 +0530)]
mgr/dashboard: set appropriate baseline branch for applitools

All the dashboard PRs are checked against a baseline branch called
'default' in the visual regresstion testing. This will cause issues when
testing PRs in different branches. For eg: currently our master and
pacific has to save two different screenshots since the two of them
differ slightly.

Disabling the applitools logs as well because its too 'noisy'

Fixes: https://tracker.ceph.com/issues/54190
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #44477 from SMIL-Infra/fix-expire_recursive-segfault
Venky Shankar [Wed, 9 Feb 2022 06:08:58 +0000 (11:38 +0530)]
Merge pull request #44477 from SMIL-Infra/fix-expire_recursive-segfault

mds: fix seg fault in expire_recursive

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44638 from yunfeiguan/wip-53911
Venky Shankar [Wed, 9 Feb 2022 06:07:20 +0000 (11:37 +0530)]
Merge pull request #44638 from yunfeiguan/wip-53911

mds: kill session state are open when mds do ms_handle_remote_reset

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #42549 from ajarr/wip-add-volume-rename
Venky Shankar [Wed, 9 Feb 2022 06:04:32 +0000 (11:34 +0530)]
Merge pull request #42549 from ajarr/wip-add-volume-rename

mgr/volumes: Add `fs volume rename` command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44651 from nmshelke/fake-mount-53903
Venky Shankar [Wed, 9 Feb 2022 06:00:27 +0000 (11:30 +0530)]
Merge pull request #44651 from nmshelke/fake-mount-53903

mount: add option to supports fake mounts in mount.ceph

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agorgw: Erase old storage class attr when the object is rewrited using radosgw-admin. 44492/head
zhangzhiming [Wed, 9 Feb 2022 00:45:52 +0000 (08:45 +0800)]
rgw: Erase old storage class attr when the object is rewrited using radosgw-admin.

Fixes: https://tracker.ceph.com/issues/53790
Signed-off-by: zhiming zhang <zhangzhm1@chinatelecom.cn>
3 years agoMerge pull request #44813 from soumyakoduri/wip-skoduri-dbstore-obj-id
Soumya Koduri [Tue, 8 Feb 2022 17:14:45 +0000 (22:44 +0530)]
Merge pull request #44813 from soumyakoduri/wip-skoduri-dbstore-obj-id

rgw/dbstore: Use Object ID to handle I/O concurrency

3 years agoMerge pull request #44678 from kipdoudou/master
Casey Bodley [Tue, 8 Feb 2022 15:27:01 +0000 (10:27 -0500)]
Merge pull request #44678 from kipdoudou/master

rgw: CopyObject works with x-amz-copy-source-if-* headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agorgw/dbstore: Handle read vs delete races 44813/head
Soumya Koduri [Thu, 3 Feb 2022 18:55:22 +0000 (00:25 +0530)]
rgw/dbstore: Handle read vs delete races

Now that tail objects are associated with objectID, they are not deleted
as part of this DeleteObj operation. Such tail objects (with no head object
in *.object.table are cleaned up later by GC thread.

To avoid races between writes/reads & GC delete, mtime is maintained for each
tail object. This mtime is updated when tail object is written and also when
its corresponding head object is deleted (like here in this case).

Also done some cleanup.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agoMerge pull request #44930 from rishabh-d-dave/py-req-fix
Laura Flores [Tue, 8 Feb 2022 13:53:02 +0000 (07:53 -0600)]
Merge pull request #44930 from rishabh-d-dave/py-req-fix

3 years agoMerge pull request #44667 from dvanders/dvanders_oft
Venky Shankar [Tue, 8 Feb 2022 13:06:56 +0000 (18:36 +0530)]
Merge pull request #44667 from dvanders/dvanders_oft

mds: mds_oft_prefetch_dirfrags default to false

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44922 from ronen-fr/wip-rf-fix-bescrub-test
Ronen Friedman [Tue, 8 Feb 2022 12:59:10 +0000 (14:59 +0200)]
Merge pull request #44922 from ronen-fr/wip-rf-fix-bescrub-test

osd/scrub: fix unintended changes to scrub (cluster)logs

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
3 years agoMerge pull request #44927 from dang/wip-dang-cloud-tier
Daniel Gryniewicz [Tue, 8 Feb 2022 12:29:25 +0000 (07:29 -0500)]
Merge pull request #44927 from dang/wip-dang-cloud-tier

RGW - Move early return

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
3 years agoosd/scrub: fix unintended changes to scrub (cluster)logs 44922/head
Ronen Friedman [Sun, 30 Jan 2022 15:23:39 +0000 (15:23 +0000)]
osd/scrub: fix unintended changes to scrub (cluster)logs

OSD logs and cluster logs are monitored by some scrub tests:
some specific strings are required to either appear or not appear in
the logs. The Scrubber backend PR has unintentionally modified some
of these logs, and here we restore the exact logs text.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
3 years agorbd: mark optional positional arguments as such in help output 44937/head
Ilya Dryomov [Tue, 8 Feb 2022 09:11:49 +0000 (10:11 +0100)]
rbd: mark optional positional arguments as such in help output

Currently at least five commands have optional positional arguments.

Overloading po::value<std::string>()->default_value("") for this
is a bit sneaky but nothing better fits into the existing Shell.cc
framework.

Note that strictly speaking "[<interval>] [<start-time>]" should be
"[<interval> [<start-time>]]" but we aren't doing that here because
"ceph" command doesn't do it either.

Fixes: https://tracker.ceph.com/issues/54191
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agomonitoring: mention PyYAML only once in requirements 44930/head
Rishabh Dave [Mon, 7 Feb 2022 18:44:42 +0000 (00:14 +0530)]
monitoring: mention PyYAML only once in requirements

Following error occurs while running "sudo install-deps.sh" -
ERROR: Double requirement given: PyYAML==6.0 (from -r requirements-lint.txt (line 5)) (already in pyyaml (from -r requirements-alerts.txt (line 1)), name='PyYAML')

PyYAML is mentioned twice as a requirement. It is mentioned once in both
the following files -
monitoring/ceph-mixin/requirements-lint.txt
monitoring/ceph-mixin/requirements-alerts.txt

These requirements were added in commits
44d3e4c264506154373ffaeb13d6c924c580e6b5 and
4750ac0d7766a8a089adf073415af0ac0d3f81d9.

Fixes: https://tracker.ceph.com/issues/54185
Signed-off-by: Rishabh Dave <ridave@redhat.com>
3 years agoMerge pull request #44921 from cyx1231st/wip-seastore-cleanup-lba
Samuel Just [Tue, 8 Feb 2022 05:37:19 +0000 (21:37 -0800)]
Merge pull request #44921 from cyx1231st/wip-seastore-cleanup-lba

crimson/os/seastore/lba_manager: consolidate logs

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44916 from joke-lee/dbstore-test-init-env
Soumya Koduri [Tue, 8 Feb 2022 04:00:41 +0000 (09:30 +0530)]
Merge pull request #44916 from joke-lee/dbstore-test-init-env

rgw: fix dbstore test init env wrong

3 years agocrimson/os/seastore/lba_manager: introduce subsys seastore_lba_details 44921/head
Yingxin Cheng [Mon, 7 Feb 2022 10:46:40 +0000 (18:46 +0800)]
crimson/os/seastore/lba_manager: introduce subsys seastore_lba_details

We will share the lba tree implementation, so classify the related logs
and rename to seastore_lba_details for now.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/lba_manager: consolidate logs
Yingxin Cheng [Mon, 7 Feb 2022 10:50:47 +0000 (18:50 +0800)]
crimson/os/seastore/lba_manager: consolidate logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44713 from aclamk/fix-aclamk-vselector-53906
Yuri Weinstein [Mon, 7 Feb 2022 22:49:05 +0000 (14:49 -0800)]
Merge pull request #44713 from aclamk/fix-aclamk-vselector-53906

[BlueStore] Fix problem with volume selector

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 years agoMerge pull request #44261 from Matan-B/wip-matanb-c-msgr-network
Samuel Just [Mon, 7 Feb 2022 21:09:12 +0000 (13:09 -0800)]
Merge pull request #44261 from Matan-B/wip-matanb-c-msgr-network

test/crimson: Inject socket level random delays/failures msgr unit test

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44856 from cbodley/wip-54103
Ali Maredia [Mon, 7 Feb 2022 21:02:40 +0000 (16:02 -0500)]
Merge pull request #44856 from cbodley/wip-54103

qa/rgw: install s3cmd with extra_system_packages

Reviewed-by: Ali Maredia <amaredia@redhat.com>
3 years agoMerge pull request #44886 from holzman/bugfix-54128
Ilya Dryomov [Mon, 7 Feb 2022 20:41:06 +0000 (21:41 +0100)]
Merge pull request #44886 from holzman/bugfix-54128

krbd: return error when no initial monitor address found

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agokrbd: return error when no initial monitor address found 44886/head
Burt Holzman [Wed, 2 Feb 2022 15:18:52 +0000 (09:18 -0600)]
krbd: return error when no initial monitor address found

Since we filter monitor addresses based on ms_mode, check that at
least one address was found.

Otherwise, we mismatch arguments when calling sysfs/add_single_major
which emits a misleading error message to dmesg:

  libceph: resolve 'name=user1' (ret=-3): failed
  libceph: parse_ips bad ip 'name=user1,key=client.user1'

Fixes: https://tracker.ceph.com/issues/54128
Signed-off-by: Burt Holzman <burt@fnal.gov>
3 years agoRGW - Move early return 44927/head
Daniel Gryniewicz [Mon, 7 Feb 2022 15:08:02 +0000 (10:08 -0500)]
RGW - Move early return

When transitioning an object to cloud, there was an early return,
skipping the removal of the cloud target.  Fix this to be in the right
place.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #44825 from rhcs-dashboard/rgw-dashboard-test-hanging
Ernesto Puerta [Mon, 7 Feb 2022 13:07:43 +0000 (14:07 +0100)]
Merge pull request #44825 from rhcs-dashboard/rgw-dashboard-test-hanging

mgr/dashboard: fix for cephadm e2e failing because of rgw commands getting stuck

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44920 from rhcs-dashboard/cephadm-bootstrap-fix
Ernesto Puerta [Mon, 7 Feb 2022 13:02:20 +0000 (14:02 +0100)]
Merge pull request #44920 from rhcs-dashboard/cephadm-bootstrap-fix

cephadm: change shared_folder directory for prometheus and grafana

Reviewed-by: MrFreezeex <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agocrimson/os/seastore/lba_manager: convert to seastore logging
Yingxin Cheng [Mon, 7 Feb 2022 02:32:52 +0000 (10:32 +0800)]
crimson/os/seastore/lba_manager: convert to seastore logging

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocephadm: change shared_folder directory for prometheus and grafana 44920/head
Nizamudeen A [Mon, 7 Feb 2022 10:53:29 +0000 (16:23 +0530)]
cephadm: change shared_folder directory for prometheus and grafana

After https://github.com/ceph/ceph/pull/44059 the monitoring/prometheus
and monitoring/grafana/dashboards directories are changed to
monitoring/ceph-mixins. That broke the shared_folders in the cephadm
bootstrap script.

Changed all the instances of monitoring/prometheus and
monitoring/grafana/dashboards to monitoring/ceph-mixins

Also, renaming all the instances of prometheus_alerts.yaml to
prometheus_alerts.yml.

Fixes: https://tracker.ceph.com/issues/54176
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agocrimson/os/seastore/lba_manager: move lba_tree_inner_stats into LBAManager
Yingxin Cheng [Mon, 7 Feb 2022 10:34:40 +0000 (18:34 +0800)]
crimson/os/seastore/lba_manager: move lba_tree_inner_stats into LBAManager

Cleanup, change stats from LBATree static member to BtreeLBAManager
class member.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agorgw: fix dbstore test init env wrong 44916/head
yuliyang_yewu [Mon, 7 Feb 2022 06:23:19 +0000 (14:23 +0800)]
rgw: fix dbstore test init env wrong

fix https://tracker.ceph.com/issues/54174

Signed-off-by: yuliyang_yewu <yuliyang_yewu@cmss.chinamobile.com>
3 years agoMerge pull request #44503 from lxbsz/typo1
Venky Shankar [Mon, 7 Feb 2022 04:33:31 +0000 (10:03 +0530)]
Merge pull request #44503 from lxbsz/typo1

mds: fix a typo of inode version related comment

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agorgw/dbstore: Create GC Thread to delete stale objects
Soumya Koduri [Wed, 26 Jan 2022 19:42:44 +0000 (01:12 +0530)]
rgw/dbstore: Create GC Thread to delete stale objects

Create a gc thread to cleanup the stale tail objects data

XXX: handle read + delete usecase, simple approach could be
to use locks or sqlite transactions in GC

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agorgw/dbstore: Use Object ID to handle racing writes
Soumya Koduri [Thu, 13 Jan 2022 20:44:19 +0000 (02:14 +0530)]
rgw/dbstore: Use Object ID to handle racing writes

Create unique ID for each object upload which will be
atomically updated in the head object at the end. This will
prevent data corruption during concurrent writes.
Incase of Multipart Uploads, upload_id is used as ObjectID.

XXX: The stale or obsolete tail data needs to be deleted

Also addressed invalid usage of CephContext in dbstore tests.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
3 years agoMerge pull request #44842 from idryomov/wip-krbd-rxbounce-option
Ilya Dryomov [Sun, 6 Feb 2022 19:37:31 +0000 (20:37 +0100)]
Merge pull request #44842 from idryomov/wip-krbd-rxbounce-option

rbd: recognize rxbounce map option

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43910 from sunnyku/wip-rbd-snap-sched
Ilya Dryomov [Sun, 6 Feb 2022 19:32:09 +0000 (20:32 +0100)]
Merge pull request #43910 from sunnyku/wip-rbd-snap-sched

mgr/rbd_support: fix schedule remove

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agotest/crimson: inject Socket layer delays/failures msgr utest 44261/head
Matan Breizman [Mon, 29 Nov 2021 14:21:40 +0000 (14:21 +0000)]
test/crimson: inject Socket layer delays/failures msgr utest

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
3 years agoMerge pull request #44893 from cfsnyder/wip-54130-oplog-segfault
Casey Bodley [Sat, 5 Feb 2022 13:54:57 +0000 (08:54 -0500)]
Merge pull request #44893 from cfsnyder/wip-54130-oplog-segfault

rgw: fix segfault in OpsLogRados::log when realm is reloaded

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44851 from batrick/i54081
Venky Shankar [Sat, 5 Feb 2022 12:34:29 +0000 (18:04 +0530)]
Merge pull request #44851 from batrick/i54081

mds: add inline flag to MDS bootstrap compat in MDSMap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44887 from josephsawaya/vstart-dev
Samuel Just [Sat, 5 Feb 2022 05:26:11 +0000 (21:26 -0800)]
Merge pull request #44887 from josephsawaya/vstart-dev

src/vstart.sh: Add --no-restart to vstart & add mkfs logs to osd.stdout

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agorgw: CopyObject works with x-amz-copy-source-if-* headers 44678/head
wanghao72 [Thu, 20 Jan 2022 07:57:29 +0000 (15:57 +0800)]
rgw: CopyObject works with x-amz-copy-source-if-* headers

CopyObject api support condition headers, eg x-amz-copy-source-if-match, while radosgw miss out the 'source' keyword
Fixes: https://tracker.ceph.com/issues/53945
Signed-off-by: Wang Hao <wanghao72@baidu.com>
3 years agosrc/vstart.sh: Add --no-restart to vstart & add mkfs logs to osd.stdout 44887/head
Joseph Sawaya [Thu, 3 Feb 2022 16:46:44 +0000 (11:46 -0500)]
src/vstart.sh: Add --no-restart to vstart & add mkfs logs to osd.stdout

The output of mkfs wasn't also being included in the OSD's log before this which
can make it more difficult to debug issues with mkfs.

ceph-run restarting every 5 seconds can make it difficult to read the osd's stdout.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
3 years agoMerge pull request #44868 from neha-ojha/wip-move-to-stream
Neha Ojha [Fri, 4 Feb 2022 19:56:08 +0000 (11:56 -0800)]
Merge pull request #44868 from neha-ojha/wip-move-to-stream

qa/distros: remove centos8

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #44866 from yuvalif/wip-yuval-fix-datalog-list
Casey Bodley [Fri, 4 Feb 2022 19:55:07 +0000 (14:55 -0500)]
Merge pull request #44866 from yuvalif/wip-yuval-fix-datalog-list

rgw/admin: fix radosgw-admin datalog list max-entries issue

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agoMerge pull request #44386 from ivancich/wip-reshard-tentative-goal
Casey Bodley [Fri, 4 Feb 2022 19:54:36 +0000 (14:54 -0500)]
Merge pull request #44386 from ivancich/wip-reshard-tentative-goal

rgw: in bucket reshard list, clarify new num shards is tentative

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
3 years agoMerge pull request #42330 from SWJTU-ZhangLei/wip-rgw-postobj
Casey Bodley [Fri, 4 Feb 2022 19:50:45 +0000 (14:50 -0500)]
Merge pull request #42330 from SWJTU-ZhangLei/wip-rgw-postobj

rgw: RGWPostObj::execute() may lost data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44743 from linuxbox2/wip-rgwlc-lcgetv
Casey Bodley [Fri, 4 Feb 2022 19:50:27 +0000 (14:50 -0500)]
Merge pull request #44743 from linuxbox2/wip-rgwlc-lcgetv

rgwlc: fix compat-decoding of cls_rgw_lc_get_entry_ret

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44137 from ivancich/wip-fix-reshard-cancel
Casey Bodley [Fri, 4 Feb 2022 19:50:10 +0000 (14:50 -0500)]
Merge pull request #44137 from ivancich/wip-fix-reshard-cancel

rgw: "reshard cancel" errors with "invalid argument"

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44892 from trociny/wip-54119
Casey Bodley [Fri, 4 Feb 2022 19:49:47 +0000 (14:49 -0500)]
Merge pull request #44892 from trociny/wip-54119

rgw: check bucket shard init status in RGWRadosBILogTrimCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoqa/suites/krbd: add legacy+rxbounce and crc+rxbounce coverage 44842/head
Ilya Dryomov [Mon, 31 Jan 2022 13:08:26 +0000 (14:08 +0100)]
qa/suites/krbd: add legacy+rxbounce and crc+rxbounce coverage

For basic, rbd and rbd-nomount subsuites, replace legacy and crc
facets with "legacy or legacy+rxbounce" and "crc or crc+rxbounce"
facets (chosen at random).

For fsx, singleton and thrash subsuites, add legacy+rxbounce and
crc+rxbounce facets and drop prefer-crc facet.  The expected behaviour
of the latter depends on cluster configuration and should be tested
separately.

The total number of jobs remains the same.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoqa: krbd rxbounce test
Ilya Dryomov [Thu, 27 Jan 2022 16:15:01 +0000 (17:15 +0100)]
qa: krbd rxbounce test

Lives in its own directory since ms_mode doesn't need to be permuted
here.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agorbd: recognize rxbounce map option
Ilya Dryomov [Wed, 26 Jan 2022 18:36:26 +0000 (19:36 +0100)]
rbd: recognize rxbounce map option

Fixes: https://tracker.ceph.com/issues/54063
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #44563 from benhanokh/NCB_new_alloc_map
Neha Ojha [Fri, 4 Feb 2022 17:31:16 +0000 (09:31 -0800)]
Merge pull request #44563 from benhanokh/NCB_new_alloc_map

BlueStore:NCB:Bug-Fix for recovery code with shared blobs

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
3 years agoMerge pull request #44796 from pcuzner/remove-old-mib
Ernesto Puerta [Fri, 4 Feb 2022 16:42:08 +0000 (17:42 +0100)]
Merge pull request #44796 from pcuzner/remove-old-mib

monitoring: remove old MIB

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44857 from rhcs-dashboard/test-doc-dash-cephadm
Ernesto Puerta [Fri, 4 Feb 2022 16:35:54 +0000 (17:35 +0100)]
Merge pull request #44857 from rhcs-dashboard/test-doc-dash-cephadm

doc: update kcli test env documentation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44059 from MrFreezeex/mixins
Ernesto Puerta [Fri, 4 Feb 2022 16:28:44 +0000 (17:28 +0100)]
Merge pull request #44059 from MrFreezeex/mixins

mgr/dashboard: monitoring: refactor into ceph-mixin

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
3 years agoMerge pull request #44862 from andriytk/fix-put-memleak2
Daniel Gryniewicz [Fri, 4 Feb 2022 16:24:58 +0000 (11:24 -0500)]
Merge pull request #44862 from andriytk/fix-put-memleak2

rgw_sal_motr: fix possible memleak on put

3 years agoqa/workunits/rbd: improve schedule add/remove cli test 43910/head
Sunny Kumar [Wed, 19 Jan 2022 13:15:52 +0000 (13:15 +0000)]
qa/workunits/rbd: improve schedule add/remove cli test

This patch adds few tests to cover schedule add/remove with invalid
inputs.

Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
3 years agomgr/rbd_support: fix schedule remove
Sunny Kumar [Fri, 12 Nov 2021 16:27:55 +0000 (16:27 +0000)]
mgr/rbd_support: fix schedule remove

Issue:

If we provide a random string in the schedule remove
command the entire schedule at specified level gets
removed.

Fixes: https://tracker.ceph.com/issues/53250
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
3 years agoMerge pull request #44348 from soumyakoduri/wip-skoduri-lc-teuthology
Casey Bodley [Fri, 4 Feb 2022 16:04:13 +0000 (11:04 -0500)]
Merge pull request #44348 from soumyakoduri/wip-skoduri-lc-teuthology

rgw/qa: Add test suite for lifecycle cases

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agomgr/dashboard: fix for cephadm e2e failing because of rgw commands getting stuck 44825/head
Nizamudeen A [Fri, 28 Jan 2022 13:53:31 +0000 (19:23 +0530)]
mgr/dashboard: fix for cephadm e2e failing because of rgw commands getting stuck

Delaying the rgw service creation in the tests until the cluster is
healthy

also changing the node_ip_offset to 110 because in the jenkins I saw

Fixes: https://tracker.ceph.com/issues/54030
Signed-off-by: Nizamudeen A <nia@redhat.com>
3 years agoos/BlueStore: NCB fixes recovery code with shared blobs 44563/head
Gabriel BenHanokh [Wed, 12 Jan 2022 12:48:27 +0000 (14:48 +0200)]
os/BlueStore: NCB fixes recovery code with shared blobs

Replaces the BitmapAllocator used by NCB Recovery code with a dedicated SimpleBitmap.
The SimpleBitmap allows for bits to be set multiple times without any adverse effect.
This is needed beacuse shared-blobs will report the same allocation multiple times.

Fixes: https://tracker.ceph.com/issues/53678
Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
3 years agoMerge pull request #44003 from zdover23/wip-doc-2021-11-17-start-documenting-ceph...
Neha Ojha [Thu, 3 Feb 2022 23:36:22 +0000 (15:36 -0800)]
Merge pull request #44003 from zdover23/wip-doc-2021-11-17-start-documenting-ceph-python-pip-update

doc/start: update python-pip to python3-pip

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44560 from jdurgin/wip-edit-on-github
Neha Ojha [Thu, 3 Feb 2022 23:28:16 +0000 (15:28 -0800)]
Merge pull request #44560 from jdurgin/wip-edit-on-github

doc/conf.py: use links to the 'edit' page for edit on github

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44888 from josephsawaya/zns-dev
Samuel Just [Thu, 3 Feb 2022 23:00:47 +0000 (15:00 -0800)]
Merge pull request #44888 from josephsawaya/zns-dev

crimson/os/seastore: change segment_off_t to seastore_off_t in ZNSSegmentManager

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge PR #44691 into master
Patrick Donnelly [Thu, 3 Feb 2022 21:40:07 +0000 (16:40 -0500)]
Merge PR #44691 into master

* refs/pull/44691/head:
doc: document PI .qa symlinks
qa: fix or add missing .qa links
qa: move .qa terminator

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agorgw: fix segfault in OpsLogRados::log when realm is reloaded 44893/head
Cory Snyder [Thu, 3 Feb 2022 19:48:05 +0000 (14:48 -0500)]
rgw: fix segfault in OpsLogRados::log when realm is reloaded

We weren't previously handling the deallocation of the store when
a realm was reloaded. Now passing a const reference to the pointer.

Fixes: https://tracker.ceph.com/issues/54130
Signed-off-by: Cory Snyder <csnyder@iland.com>
3 years agorgw_sal_motr: fix possible memleak on put 44862/head
Andriy Tkachuk [Wed, 2 Feb 2022 11:25:59 +0000 (11:25 +0000)]
rgw_sal_motr: fix possible memleak on put

Currently, the MotrAtomicWriter::cleanup() is called from
MotrAtomicWriter::commit(), which may not be called at all
by rgw in case of md5 checksum failure.

Solution: call cleanup() from process() when data is zero.
rgw calls Writer::process(data, off) with zero data at the
end of the loop to allow writes to flush the data. From:
src/rgw/rgw_op.cc:RGWPutObj::execute():

      op_ret = filter->process(std::move(data), ofs);
      ...
      ofs += len;
    } while (len > 0);

    // flush any data in filters
    op_ret = filter->process({}, ofs);

Signed-off-by: Andriy Tkachuk <andriy.tkachuk@seagate.com>
Reviewed-by: Sining Wu <sining.wu@seagate.com>