]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agorgw/multisite: metadata sync only retries on errors 44078/head
Casey Bodley [Tue, 23 Nov 2021 20:44:03 +0000 (15:44 -0500)]
rgw/multisite: metadata sync only retries on errors

in 866d66b8749b28ec626a8d0adba3d14fdd8abead, metadata sync was fixed to
retry on error codes other than EAGAIN/ECANCELED. but this change caused
us to retry on success as well, which means we send 10 GET requests for
each piece of metadata, and write it to rados 10 times

Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44427 from lxbsz/client_cleanup
Venky Shankar [Thu, 13 Jan 2022 15:04:54 +0000 (20:34 +0530)]
Merge pull request #44427 from lxbsz/client_cleanup

client: remove useless Lx cap check

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44451 from lxbsz/wip-53750
Venky Shankar [Thu, 13 Jan 2022 15:03:58 +0000 (20:33 +0530)]
Merge pull request #44451 from lxbsz/wip-53750

mds: directly return just after responding the link request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44561 from cbodley/wip-51727
Casey Bodley [Thu, 13 Jan 2022 14:38:49 +0000 (09:38 -0500)]
Merge pull request #44561 from cbodley/wip-51727

qa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #44538 from dang/wip-dang-zipper-perf
Daniel Gryniewicz [Thu, 13 Jan 2022 14:09:33 +0000 (09:09 -0500)]
Merge pull request #44538 from dang/wip-dang-zipper-perf

RGW Zipper - don't load stats for every bucket load

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44002 from JoshSalomon/wip-primary-balancer
Laura Flores [Thu, 13 Jan 2022 13:45:54 +0000 (07:45 -0600)]
Merge pull request #44002 from JoshSalomon/wip-primary-balancer

3 years agomds: directly return just after responding the link request 44451/head
Xiubo Li [Tue, 4 Jan 2022 03:18:53 +0000 (11:18 +0800)]
mds: directly return just after responding the link request

Fixes: https://tracker.ceph.com/issues/53750
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #43286 from lxbsz/improve_setattr
Venky Shankar [Thu, 13 Jan 2022 12:53:27 +0000 (18:23 +0530)]
Merge pull request #43286 from lxbsz/improve_setattr

client: buffer the truncate if we have the Fx caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoclient: remove useless Lx cap check 44427/head
Xiubo Li [Thu, 30 Dec 2021 07:03:35 +0000 (15:03 +0800)]
client: remove useless Lx cap check

Once here the new_caps must have the 'Ls' caps, the extra check
for 'Lsx' makes no sense.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #44229 from lxbsz/mds-buffix
Venky Shankar [Thu, 13 Jan 2022 12:46:13 +0000 (18:16 +0530)]
Merge pull request #44229 from lxbsz/mds-buffix

mds: remove the duplicated or incorrect respond

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44397 from lxbsz/wip-53726
Venky Shankar [Thu, 13 Jan 2022 12:45:24 +0000 (18:15 +0530)]
Merge pull request #44397 from lxbsz/wip-53726

mds: dump tree '/' when the path is empty

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44422 from lxbsz/wip-51705
Venky Shankar [Thu, 13 Jan 2022 12:44:14 +0000 (18:14 +0530)]
Merge pull request #44422 from lxbsz/wip-51705

qa: do not use any time related suffix for *_op_timeouts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43995 from TRYTOBE8TME/wip-rgw-kafka-teuth-cleanup
Yuval Lifshitz [Thu, 13 Jan 2022 09:57:03 +0000 (11:57 +0200)]
Merge pull request #43995 from TRYTOBE8TME/wip-rgw-kafka-teuth-cleanup

qa/tasks: Checking for kafka cleanup

3 years agoosd, tools: refactor OSDMap::calc_pg_upmaps (simplify the code) 44002/head
Josh Salomon [Thu, 13 Jan 2022 02:23:07 +0000 (02:23 +0000)]
osd, tools: refactor OSDMap::calc_pg_upmaps (simplify the code)

This is the first commit in a series of commits that aims at adding a primary balancer to Ceph and improving the current upmap balancer functionality. This first commit focuses on simplifying (refactoring) the code of `calc_pg_upmaps` so it is easier to change in the future. This PR keeps the existing functionality as-is and does not change anything but the code structure.

As part of the work is major refactoring of OSDMap::calc_pg_upmaps, the first thing is adding an --upmap-seed param to osdmaptool so test results can be compared without the random factor.

Other changes made:
    - Divided sections of `OSDMap::calc_pg_upmaps` into their own separate functions
    - Renamed tmp to tmp_osd_map
    - Changed all the occurances of 'first' and 'second' in the function to more meaningful names.

Signed-off-by: Josh Salomon <josh.salomon@gmail.com>
3 years agoMerge pull request #43299 from markhpc/wip-age-binning-rebase-20210923
Yuri Weinstein [Thu, 13 Jan 2022 00:54:23 +0000 (16:54 -0800)]
Merge pull request #43299 from markhpc/wip-age-binning-rebase-20210923

common/PriorityCache: Updated Implementation of Cache Age Binning

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoqa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist 44561/head
Casey Bodley [Wed, 12 Jan 2022 19:07:26 +0000 (14:07 -0500)]
qa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist

and cover rgw/singleton suite

Fixes: https://tracker.ceph.com/issues/51727
Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43494 from majianpeng/enable-test-librbd-BlockGuard
Ilya Dryomov [Wed, 12 Jan 2022 20:50:00 +0000 (21:50 +0100)]
Merge pull request #43494 from majianpeng/enable-test-librbd-BlockGuard

test/librbd: re-enable BlockGuard test

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #44478 from cyx1231st/wip-crimson-improve-log-3
Samuel Just [Wed, 12 Jan 2022 20:27:14 +0000 (12:27 -0800)]
Merge pull request #44478 from cyx1231st/wip-crimson-improve-log-3

crimson/os/seastore/../segment_manager: improve logs and validations

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #44536 from yuvalif/wip-yuval-dynamic-reshard
Casey Bodley [Wed, 12 Jan 2022 19:14:41 +0000 (14:14 -0500)]
Merge pull request #44536 from yuvalif/wip-yuval-dynamic-reshard

rgw: fix dynamic reshard happening during user stats sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoRGW Zipper - don't load stats for every bucket load 44538/head
Daniel Gryniewicz [Tue, 11 Jan 2022 15:50:00 +0000 (10:50 -0500)]
RGW Zipper - don't load stats for every bucket load

This was a side-effect of consolidating the Zipper API, and resulted in
a large performance hit.  Stats are only needed if they are requested,
so don't load them every time.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #39440 from pdvian/wip-warn-filestore-osds
Yuri Weinstein [Wed, 12 Jan 2022 16:49:02 +0000 (08:49 -0800)]
Merge pull request #39440 from pdvian/wip-warn-filestore-osds

mon/OSDMonitor, osd: Add warning on filestore deprecation and force use of wpq scheduler for filestore OSDs

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #41778 from felixhuettner/fix_subdir_name
Daniel Gryniewicz [Wed, 12 Jan 2022 14:22:04 +0000 (09:22 -0500)]
Merge pull request #41778 from felixhuettner/fix_subdir_name

rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

3 years agoMerge pull request #38234 from inspur-wyq/wip-copy-obj-check-size
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:48 +0000 (09:21 -0500)]
Merge pull request #38234 from inspur-wyq/wip-copy-obj-check-size

rgw : check the object size when copy obj

3 years agoMerge pull request #38532 from Rjerk/wip-empty-tagset
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:35 +0000 (09:21 -0500)]
Merge pull request #38532 from Rjerk/wip-empty-tagset

rgw: an empty tagset is allowed by S3

3 years agoMerge pull request #40573 from Huber-ming/rgw_admin
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:15 +0000 (09:21 -0500)]
Merge pull request #40573 from Huber-ming/rgw_admin

rgw: delete abbreviation for option "--new-uid"

3 years agoMerge pull request #40575 from Huber-ming/rgw_admin-f
Daniel Gryniewicz [Wed, 12 Jan 2022 14:20:49 +0000 (09:20 -0500)]
Merge pull request #40575 from Huber-ming/rgw_admin-f

radosgw-admin: delete the abbreviation of option "--infile"

3 years agoMerge pull request #44500 from idryomov/wip-rbd-test-group-leak
Ilya Dryomov [Wed, 12 Jan 2022 11:56:09 +0000 (12:56 +0100)]
Merge pull request #44500 from idryomov/wip-rbd-test-group-leak

test/librbd: fix group_info.name leaks in TestGroup.add_image

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44490 from liu-chunmei/crimson-fix-aligned
Liu-Chunmei [Wed, 12 Jan 2022 06:05:22 +0000 (22:05 -0800)]
Merge pull request #44490 from liu-chunmei/crimson-fix-aligned

crimson: fix assert_aligned(size) in trim_data_reservation

reviewed by: Samuel Just <sjust@redhat.com> , Yingxin <yingxin.cheng@intel.com>

3 years agocrimson/os/seastore/../segment_manager: add more validations 44478/head
Yingxin Cheng [Thu, 6 Jan 2022 08:03:20 +0000 (16:03 +0800)]
crimson/os/seastore/../segment_manager: add more validations

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: consolidate logs with structured level and...
Yingxin Cheng [Thu, 6 Jan 2022 08:01:28 +0000 (16:01 +0800)]
crimson/os/seastore/../segment_manager: consolidate logs with structured level and format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: cleanup device_id usage
Yingxin Cheng [Thu, 6 Jan 2022 02:26:20 +0000 (10:26 +0800)]
crimson/os/seastore/../segment_manager: cleanup device_id usage

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: pretty print data structures
Yingxin Cheng [Wed, 5 Jan 2022 14:56:13 +0000 (22:56 +0800)]
crimson/os/seastore/../segment_manager: pretty print data structures

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: convert to seastore logging
Yingxin Cheng [Wed, 5 Jan 2022 05:32:46 +0000 (13:32 +0800)]
crimson/os/seastore/../segment_manager: convert to seastore logging

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: suppress compile warning about unused logger
Yingxin Cheng [Wed, 5 Jan 2022 03:22:09 +0000 (11:22 +0800)]
crimson/os/seastore/../segment_manager: suppress compile warning about unused logger

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44532 from rzarzynski/wip-crimson-fix-test-runner
Yingxin [Wed, 12 Jan 2022 01:23:53 +0000 (09:23 +0800)]
Merge pull request #44532 from rzarzynski/wip-crimson-fix-test-runner

test/crimson: fix a race condition in SeastarRunner

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43593 from ljflores/wip-rocksdb
Neha Ojha [Wed, 12 Jan 2022 00:31:41 +0000 (16:31 -0800)]
Merge pull request #43593 from ljflores/wip-rocksdb

mgr: expose rocksdb version number for use in telemetry

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agocrimson: fix assert_aligned(size) in trim_data_reservation 44490/head
chunmei-liu [Tue, 11 Jan 2022 03:44:43 +0000 (19:44 -0800)]
crimson: fix assert_aligned(size) in trim_data_reservation

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
signed-off-by: Samuel Just <sjust@redhat.com>

3 years agomgr/telemetry: add the rocksdb version number to telemetry 43593/head
Laura Flores [Tue, 4 Jan 2022 22:54:33 +0000 (22:54 +0000)]
mgr/telemetry: add the rocksdb version number to telemetry

Capturing the RocksDB version number in Telemetry would allow us to check that users are using the appropriate RocksDB version for their Ceph cluster. For instance, if a user is working in a Pacific cluster, but their RocksDB version is meant for Nautilus, that might be a problem.

It is strucured as "rocksdb_stats" --> "version" in anticipation of more stats that can will be added under "rocksdb_stats".

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr: expose rocksdb version number in the mgr module
Laura Flores [Wed, 22 Dec 2021 23:09:25 +0000 (23:09 +0000)]
mgr: expose rocksdb version number in the mgr module

It is only necessary here to link the rocksdb include directory
since the mgr simply needs access to the rocksdb version numbers.

Signed-off-by: Laura Flores <lflores@redhat.com>
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Co-authored-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #43794 from aclamk/wip-bluefs-fine-grain-locking-4
Neha Ojha [Tue, 11 Jan 2022 21:59:16 +0000 (13:59 -0800)]
Merge pull request #43794 from aclamk/wip-bluefs-fine-grain-locking-4

os/bluestore: BlueFS fine grain locking

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agocommon/PriorityCache: Implement Cache Age Binning 43299/head
Mark Nelson [Tue, 13 Oct 2020 23:24:18 +0000 (23:24 +0000)]
common/PriorityCache: Implement Cache Age Binning

Signed-off-by: Mark Nelson <mnelson@redhat.com>
3 years agoMerge pull request #44537 from pdvian/wip-typo-fix
Casey Bodley [Tue, 11 Jan 2022 20:50:34 +0000 (15:50 -0500)]
Merge pull request #44537 from pdvian/wip-typo-fix

rgw: Fix typo error for stat_remote_obj

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44496 from idryomov/wip-diff-iterate-parent-fix
Ilya Dryomov [Tue, 11 Jan 2022 19:28:19 +0000 (20:28 +0100)]
Merge pull request #44496 from idryomov/wip-diff-iterate-parent-fix

librbd: restore diff-iterate include_parent functionality in fast-diff mode

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agorgw: Fix typo error for stat_remote_obj 44537/head
Prashant D [Tue, 11 Jan 2022 18:58:20 +0000 (13:58 -0500)]
rgw: Fix typo error for stat_remote_obj

Signed-off-by: Prashant D <pdhange@redhat.com>
3 years agorgw: fix dynamic reshard happening during user stats sync 44536/head
Yuval Lifshitz [Mon, 10 Jan 2022 16:55:50 +0000 (18:55 +0200)]
rgw: fix dynamic reshard happening during user stats sync

this is a regression caused by the following commit:
https://github.com/ceph/ceph/commit/72d1a363263cf707d022ee756122236ba175cda2

to verify use the following instructions:
https://gist.github.com/yuvalif/d526c0a3a4c5b245b9e951a6c5a10517

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #44483 from idryomov/wip-diff-iterate-offset-fix
Ilya Dryomov [Tue, 11 Jan 2022 17:59:12 +0000 (18:59 +0100)]
Merge pull request #44483 from idryomov/wip-diff-iterate-offset-fix

librbd: diff-iterate reports incorrect offsets in fast-diff mode

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44355 from sseshasa/wip-fix-enhance-mclock-logging
Yuri Weinstein [Tue, 11 Jan 2022 17:32:12 +0000 (09:32 -0800)]
Merge pull request #44355 from sseshasa/wip-fix-enhance-mclock-logging

osd:  Enhance mclockscheduler logging and formatted dump information.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44482 from sebastian-philipp/nfs-upgrade-port-conflict
Yuri Weinstein [Tue, 11 Jan 2022 17:31:35 +0000 (09:31 -0800)]
Merge pull request #44482 from sebastian-philipp/nfs-upgrade-port-conflict

mgr/cephadm: nfs migration: avoid port conflicts

Reviewed-by: Adam King adking@redhat.com
3 years agoMerge pull request #36308 from linuxbox2/wip-rgwlc-bcktdel
Matt Benjamin [Tue, 11 Jan 2022 17:17:22 +0000 (12:17 -0500)]
Merge pull request #36308 from linuxbox2/wip-rgwlc-bcktdel

rgwlc:  remove lc entry on bucket delete

3 years agoMerge pull request #42891 from ofriedma/wip-ofriedma-rgw-qos-finale
Casey Bodley [Tue, 11 Jan 2022 16:35:05 +0000 (11:35 -0500)]
Merge pull request #42891 from ofriedma/wip-ofriedma-rgw-qos-finale

rgw: Add rgw rate limiting per user and per bucket

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #38774 from linuxbox2/wip-doc-replsupport
Casey Bodley [Tue, 11 Jan 2022 16:13:24 +0000 (11:13 -0500)]
Merge pull request #38774 from linuxbox2/wip-doc-replsupport

rgw: document S3 bucket replication support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoqa/tasks: Checking for kafka cleanup 43995/head
Kalpesh Pandya [Thu, 11 Nov 2021 06:46:16 +0000 (12:16 +0530)]
qa/tasks: Checking for kafka cleanup

Adding a sleep after running ./kafka-server-stop.sh and ./zookeeper-server-stop.sh
scripts so that nothing gets logged into the kafka logs after the sleep time.
And finally killing the process.

This resolves: https://tracker.ceph.com/issues/53220

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
3 years agotest/crimson: fix a race condition in SeastarRunner 44532/head
Radoslaw Zarzynski [Tue, 11 Jan 2022 12:09:28 +0000 (12:09 +0000)]
test/crimson: fix a race condition in SeastarRunner

This patch is supposed to fix the following problem:

```
        Start 234: unittest-seastar-errorator
216/258 Test #234: unittest-seastar-errorator ................Child aborted***Exception:   0.95 sec
WARNING: debug mode. Not for benchmarking or production
WARN  2022-01-11 10:01:32,973 [shard 0] seastar - Creation of perf_event based stall detector failed, falling back to posix timer: std::system_error (error system:13, perf_event_open() failed: Permission denied)
unittest-seastar-errorator: ../src/test/crimson/seastar_runner.h:45: int SeastarRunner::init(int, char **): Assertion `begin_signaled == true' failed.
Aborting.
Backtrace:
  0xb3397a
  0x1d19430
  0x1d1905d
  0x1b72332
  0x1b9dd95
  0x1c75fe9
  0x1c76231
  0x1c7605a
  0x7f20b7aee3bf
  /lib/x86_64-linux-gnu/libc.so.6+0x4618a
  /lib/x86_64-linux-gnu/libc.so.6+0x25858
  /lib/x86_64-linux-gnu/libc.so.6+0x25728
  /lib/x86_64-linux-gnu/libc.so.6+0x36f35
  0xc8bb88
  0xc7ec71
  /lib/x86_64-linux-gnu/libc.so.6+0x270b2
  0xafc95d
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43404 from rhcs-dashboard/angular-12-upgradee
Ernesto Puerta [Tue, 11 Jan 2022 12:02:07 +0000 (13:02 +0100)]
Merge pull request #43404 from rhcs-dashboard/angular-12-upgradee

mgr/dashboard: Update Angular version to 12

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
3 years agoMerge pull request #44088 from ceph/feature-48388-cache
Ernesto Puerta [Tue, 11 Jan 2022 11:52:41 +0000 (12:52 +0100)]
Merge pull request #44088 from ceph/feature-48388-cache

mgr: TTL cache implementation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
3 years agoMerge pull request #44294 from rhcs-dashboard/feature-bluestore-onode
Ernesto Puerta [Tue, 11 Jan 2022 10:24:21 +0000 (11:24 +0100)]
Merge pull request #44294 from rhcs-dashboard/feature-bluestore-onode

mgr/dashboard: monitoring:Implement BlueStore onode hit/miss counters into the dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44456 from jtlayton/wip-53765
Venky Shankar [Tue, 11 Jan 2022 09:09:16 +0000 (14:39 +0530)]
Merge pull request #44456 from jtlayton/wip-53765

mount.ceph: fix the handling of new-syntax device names

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoosd: Display scheduler specific info when dumping an OpSchedulerItem 44355/head
Sridhar Seshasayee [Fri, 17 Dec 2021 12:30:05 +0000 (18:00 +0530)]
osd: Display scheduler specific info when dumping an OpSchedulerItem

Implement logic to dump information relevant to the scheduler type being
employed when dumping details about an OpSchedulerItem. For e.g., the
'priority' field is relevant for the 'wpq' scheduler, but for the
'mclock_scheduler', the 'qos_cost' gives more information during debugging.

A couple of additional fields called 'qos_cost' and 'is_qos_request' are
introduced in OpSchedulerItem class. These are mainly used to facilitate
dumping of relevant information depending on the scheduler type. The
interesting points are when an item is enqueued and dequeued.

For the 'mclock_scheduler', the 'class_id' and the 'qos_cost' fields are
dumped during enqueue and dequeue op respectively. For the 'wpq' scheduler
things remain the same as before.

An additional benefit of this change is to help immediately identify the
type of scheduler being used for a given shard depending on what is dumped
in the debug messages while debugging.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoosd: Add debug logs and formatted dumps in the mClockScheduler
Sridhar Seshasayee [Tue, 14 Dec 2021 10:41:14 +0000 (16:11 +0530)]
osd: Add debug logs and formatted dumps in the mClockScheduler

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoosd: Change description of 'dump_op_pq_state' command to be more generic
Sridhar Seshasayee [Thu, 6 Jan 2022 09:58:21 +0000 (15:28 +0530)]
osd: Change description of 'dump_op_pq_state' command to be more generic

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agocommon: Introduce 'mclock' debug subsystem
Sridhar Seshasayee [Thu, 6 Jan 2022 09:56:20 +0000 (15:26 +0530)]
common: Introduce 'mclock' debug subsystem

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoMerge pull request #44410 from tchaikov/wip-cmake-cleanup
Samuel Just [Mon, 10 Jan 2022 21:36:09 +0000 (13:36 -0800)]
Merge pull request #44410 from tchaikov/wip-cmake-cleanup

cmake: detect linux/blk/zoned support

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44502 from yuvalif/fix-call-to-mrgw
Yuval Lifshitz [Mon, 10 Jan 2022 20:05:22 +0000 (22:05 +0200)]
Merge pull request #44502 from yuvalif/fix-call-to-mrgw

rgw/multisite/test: pass zero (no) ssl port to mrgw.sh

3 years agoMerge pull request #44495 from Matan-B/wip-matanb-crimson-flavor
Matan [Mon, 10 Jan 2022 18:30:18 +0000 (20:30 +0200)]
Merge pull request #44495 from Matan-B/wip-matanb-crimson-flavor

qa/suites: Crimson flavor configuration fix

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: chunmei-liu chunmei.liu@intel.com
3 years agotest/librbd: make diff-iterate clone tests exercise fast-diff mode 44496/head
Ilya Dryomov [Fri, 7 Jan 2022 12:31:08 +0000 (13:31 +0100)]
test/librbd: make diff-iterate clone tests exercise fast-diff mode

The fast-diff feature wasn't propagated to the clone so these tests
were exercising the slow list_snaps path no matter what RBD_FEATURES
value was supplied to ceph_test_librbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: restore diff-iterate include_parent functionality in fast-diff mode
Ilya Dryomov [Wed, 5 Jan 2022 19:24:40 +0000 (20:24 +0100)]
librbd: restore diff-iterate include_parent functionality in fast-diff mode

Commit 4429ed4f3f4c ("librbd: switch diff iterate API to use new snaps
list dispatch methods") removed the recursive execute() call.  The new
list_snaps method does indeed handle parent diffs internally but it is
not used in fast-diff mode.  Nothing changed there -- we still need to
load the parent object map, calculate parent object_diff_state, etc.

Fixes: https://tracker.ceph.com/issues/53787
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: stash unmodified include_parent value in DiffContext
Ilya Dryomov [Wed, 5 Jan 2022 18:45:50 +0000 (19:45 +0100)]
librbd: stash unmodified include_parent value in DiffContext

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agotest/librbd: fix group_info.name leaks in TestGroup.add_image 44500/head
Ilya Dryomov [Sat, 8 Jan 2022 13:18:32 +0000 (14:18 +0100)]
test/librbd: fix group_info.name leaks in TestGroup.add_image

Introduced in commit c04e8c3374e0 ("test/librbd: add get_group test").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agodoc: rgw: document S3 bucket replication support 38774/head
Matt Benjamin [Tue, 5 Jan 2021 20:30:23 +0000 (15:30 -0500)]
doc: rgw: document S3 bucket replication support

Support was added at Octopus.

Fixes: https://tracker.ceph.com/issues/48755
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
3 years agorgw: Add admin ops API for rate limiting 42891/head
Or Friedmann [Mon, 3 Jan 2022 16:32:16 +0000 (16:32 +0000)]
rgw: Add admin ops API for rate limiting

Add admin ops API for rate limiting and some bug fixes

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
3 years agoMerge pull request #44221 from BenoitKnecht/ceph-mon-device-metrics
Yuri Weinstein [Mon, 10 Jan 2022 15:56:29 +0000 (07:56 -0800)]
Merge pull request #44221 from BenoitKnecht/ceph-mon-device-metrics

mon: Abort device health when device not found

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
3 years agoMerge pull request #43362 from ybwang0211/pubsub_settingng_spell
Daniel Gryniewicz [Mon, 10 Jan 2022 15:10:51 +0000 (10:10 -0500)]
Merge pull request #43362 from ybwang0211/pubsub_settingng_spell

rgw:word spelling error in rgw/rgw_pubsub.h file

3 years agoMerge pull request #42115 from sandrobonazzola/patch-2
Sebastian Wagner [Mon, 10 Jan 2022 14:41:32 +0000 (15:41 +0100)]
Merge pull request #42115 from sandrobonazzola/patch-2

doc/install: ensure to open ceph-mon firewall ports

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agorgw/multisite/test: passzero (no) ssl port to mrgw.sh 44502/head
Yuval Lifshitz [Sun, 9 Jan 2022 16:27:40 +0000 (18:27 +0200)]
rgw/multisite/test: passzero (no) ssl port to mrgw.sh

in multisite tests script.
this is to fix an issue from: 1f3fbf218faa4859f02917828dbdb48be41a579a

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #44301 from yuvalif/wip-yuval-cloudevents
Yuval Lifshitz [Sun, 9 Jan 2022 07:32:53 +0000 (09:32 +0200)]
Merge pull request #44301 from yuvalif/wip-yuval-cloudevents

rgw/notifications: add cloudevents support to HTTP endpoint

3 years agoMerge pull request #44278 from fengchunsong/dpdk-tso
Kefu Chai [Sat, 8 Jan 2022 11:28:00 +0000 (19:28 +0800)]
Merge pull request #44278 from fengchunsong/dpdk-tso

msg/async/dpdk: Add an option to enable/disable TSO

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44198 from wjwithagen/fix-bluestore-get-io-size
Kefu Chai [Sat, 8 Jan 2022 04:29:21 +0000 (12:29 +0800)]
Merge pull request #44198 from wjwithagen/fix-bluestore-get-io-size

common/blkdev: add missing stubs for OSes not Linux

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44402 from inspur-wyq/wip-pybind-mgr
Yuri Weinstein [Fri, 7 Jan 2022 23:36:58 +0000 (15:36 -0800)]
Merge pull request #44402 from inspur-wyq/wip-pybind-mgr

pybind/mgr: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #44055 from ifed01/wip-ifed-cosmetics-for-optimize-rm
Yuri Weinstein [Fri, 7 Jan 2022 23:36:37 +0000 (15:36 -0800)]
Merge pull request #44055 from ifed01/wip-ifed-cosmetics-for-optimize-rm

test/store_test, os/bluestore: bunch of minor improvements/cleanups

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup
Yuri Weinstein [Fri, 7 Jan 2022 23:35:42 +0000 (15:35 -0800)]
Merge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup

cmake: cleanups about systemd and install

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #44423 from 5cs/fix-unshare-blob-test
Igor Fedotov [Fri, 7 Jan 2022 23:27:42 +0000 (02:27 +0300)]
Merge pull request #44423 from 5cs/fix-unshare-blob-test

test: fix bluestore unshare blob test case

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agoMerge pull request #44416 from yuvalif/wip-yuval-rgw-params-in-multisite
Casey Bodley [Fri, 7 Jan 2022 20:19:44 +0000 (15:19 -0500)]
Merge pull request #44416 from yuvalif/wip-yuval-rgw-params-in-multisite

rgw/multisite/test: allow passing rgw parameter to multisite script

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44020 from adk3798/host-ls-scale
Sebastian Wagner [Fri, 7 Jan 2022 10:59:04 +0000 (11:59 +0100)]
Merge pull request #44020 from adk3798/host-ls-scale

mgr/orchestrator: add filtering and count option for orch host ls

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agorgw/notifications: add cloudevents support to HTTP endpoint 44301/head
yuval Lifshitz [Mon, 13 Dec 2021 18:56:20 +0000 (20:56 +0200)]
rgw/notifications: add cloudevents support to HTTP endpoint

following the cloudevents HTTP spec:
https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md
and more specifically this aws-s3 spec:
https://github.com/cloudevents/spec/blob/main/cloudevents/adapters/aws-s3.md

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>
3 years agorgw/common: make "get" methods on RGWHTTPArgs const
yuval Lifshitz [Mon, 13 Dec 2021 16:45:19 +0000 (18:45 +0200)]
rgw/common: make "get" methods on RGWHTTPArgs const

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #44412 from cyx1231st/wip-crimson-improve-log
Yingxin [Fri, 7 Jan 2022 07:04:46 +0000 (15:04 +0800)]
Merge pull request #44412 from cyx1231st/wip-crimson-improve-log

crimson: leverage --default-log-level to configure the global log level

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44473 from johnbent/patch-1
Laura Flores [Thu, 6 Jan 2022 16:05:09 +0000 (10:05 -0600)]
Merge pull request #44473 from johnbent/patch-1

README.md: Update README.md to add link to tracker.ceph.com

3 years agoMerge pull request #43833 from melissa-kun-li/prevent-cephadm-stray-daemon-tcmurunner
Sebastian Wagner [Thu, 6 Jan 2022 16:00:00 +0000 (17:00 +0100)]
Merge pull request #43833 from melissa-kun-li/prevent-cephadm-stray-daemon-tcmurunner

mgr/cephadm: fix tcmu-runner cephadm_stray_daemon

3 years agoqa/suites: Fix crimson flavor configuration 44495/head
Matan Breizman [Thu, 6 Jan 2022 13:32:33 +0000 (13:32 +0000)]
qa/suites: Fix crimson flavor configuration

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
3 years agoMerge PR #44054 into master
Sage Weil [Thu, 6 Jan 2022 13:54:45 +0000 (08:54 -0500)]
Merge PR #44054 into master

* refs/pull/44054/head:
doc/rados/operations: document pg_num_max
mgr: set max of 32 pgs for .mgr pool
mgr/dashboard: expect pg_num_max property for pools
mon/OSDMonitor: add option --pg-num_max arg for create pool
mon/OSDMonitor: disallow setting pg_num < min or > max
mgr/pg_autoscaler: apply pg_num_max
mon: add pg_num_max pool property

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #44157 from CongMinYin/pwl-small-fixes
Ilya Dryomov [Thu, 6 Jan 2022 13:54:31 +0000 (14:54 +0100)]
Merge pull request #44157 from CongMinYin/pwl-small-fixes

librbd/cache/pwl: code cleanup and some minor fixes

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: diff-iterate reports incorrect offsets in fast-diff mode 44483/head
Ilya Dryomov [Tue, 4 Jan 2022 19:38:35 +0000 (20:38 +0100)]
librbd: diff-iterate reports incorrect offsets in fast-diff mode

If rbd_diff_iterate2() is called on an image offset that doesn't
correspond to an object boundary, the callback is invoked with an
incorrect image offset.  For example, assuming a fully allocated
image, a diff request for 806354944~57344 results in offs=807403520,
len=57344, exists=true invocation, which is ahead by 1048576 bytes.
This occurs only in fast-diff mode, for a diff request on an image
with the fast-diff feature disabled or if whole_object parameter is
set to false the invocation is correct.

This bug goes back to the introduction of fast-diff mode in commit
6d5b969d4206 ("librbd: add diff_iterate2 to API").

Fixes: https://tracker.ceph.com/issues/53784
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agomgr/cephadm: nfs migration: avoid port conflicts 44482/head
Sebastian Wagner [Thu, 6 Jan 2022 11:35:38 +0000 (12:35 +0100)]
mgr/cephadm: nfs migration: avoid port conflicts

The nfs upgrade renames the nfs spec from `nfs.ganesha-{service_id}`
to `nfs.{service_id}`. Previously we used the orphan-daemon check
to remove the old `nfs.ganesha-{service_id}` daemons. This does not work
as sometimes serve() tries to deploy the new daemons before cleaning up the
old daemons. This results in a port conflict breaking the upgrade.

Fixes: https://tracker.ceph.com/issues/53424
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agolibrbd/cache/pwl: merge multiple output "<<" calls 44157/head
Yin Congmin [Tue, 4 Jan 2022 08:09:27 +0000 (16:09 +0800)]
librbd/cache/pwl: merge multiple output "<<" calls

Follow the principle of fewer "<<" calls and higher performance.
Merge multiple output "<<" calls.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
3 years agoMerge pull request #44475 from athanatos/sjust/wip-53783
Yingxin [Thu, 6 Jan 2022 01:36:06 +0000 (09:36 +0800)]
Merge pull request #44475 from athanatos/sjust/wip-53783

crimson/os/seastore/cache: fix debug macro usage

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/cache: fix debug macro usage 44475/head
Samuel Just [Wed, 5 Jan 2022 23:46:33 +0000 (23:46 +0000)]
crimson/os/seastore/cache: fix debug macro usage

Introduced via conflict between 277e57 and 632916.

Fixes: https://tracker.ceph.com/issues/53783
Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoREADME.md: Update README.md to add link to tracker.ceph.com 44473/head
John Bent [Wed, 5 Jan 2022 16:04:40 +0000 (09:04 -0700)]
README.md: Update README.md to add link to tracker.ceph.com

I searched the existing documentation for a link to the tracker and had a hard time finding it. Other folks like myself might like it so prominently displayed.
Added a link to https://tracker.ceph.com/projects/ceph as https://tracker.ceph.com/ is basically blank and not totally intuitive how to get to issues.

Signed-off-by: John Bent <john.bent@seagate.com>
3 years agoMerge pull request #44429 from cyx1231st/wip-crimson-improve-log-2
Samuel Just [Wed, 5 Jan 2022 20:52:18 +0000 (12:52 -0800)]
Merge pull request #44429 from cyx1231st/wip-crimson-improve-log-2

crimson/os/seastore: differentiate loggers in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44420 from aisakaki/lru-cache
Samuel Just [Wed, 5 Jan 2022 20:34:24 +0000 (12:34 -0800)]
Merge pull request #44420 from aisakaki/lru-cache

crimson/os/seastore/cache: add lru

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44474 from ljflores/wip-doc-integration-tests
Neha Ojha [Wed, 5 Jan 2022 18:33:55 +0000 (10:33 -0800)]
Merge pull request #44474 from ljflores/wip-doc-integration-tests

doc/.../testing_integration_tests: fix formatting of teuthology-suite option

Reviewed-by: Neha Ojha <nojha@redhat.com>