]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocrimson/os/seastore: implement FuturizedStore::flush 44591/head
Samuel Just [Fri, 14 Jan 2022 06:53:17 +0000 (06:53 +0000)]
crimson/os/seastore: implement FuturizedStore::flush

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agocrimson: add and use FuturizedStore::flush() interface
Samuel Just [Fri, 14 Jan 2022 04:58:16 +0000 (04:58 +0000)]
crimson: add and use FuturizedStore::flush() interface

Signed-off-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44555 from cyx1231st/wip-fix-seastore-jounral-fast-submit
Samuel Just [Fri, 14 Jan 2022 01:23:37 +0000 (17:23 -0800)]
Merge pull request #44555 from cyx1231st/wip-fix-seastore-jounral-fast-submit

crimson/os/seastore/journal: fast submit if RecordSubmitter is IDLE and no pending

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44583 from mgfritch/fixup-44306-docker-count
Adam King [Fri, 14 Jan 2022 00:18:35 +0000 (19:18 -0500)]
Merge pull request #44583 from mgfritch/fixup-44306-docker-count

cephadm: increase number of docker.io occurances

Reviewed-by: Adam King <adking@redhat.com>
3 years agocephadm: increase number of docker.io occurances 44583/head
Michael Fritch [Thu, 13 Jan 2022 22:22:40 +0000 (15:22 -0700)]
cephadm: increase number of docker.io occurances

fixup for 0fe2e54db774271e4fc18b45aba36b66cbc71779

Signed-off-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44554 from jdurgin/wip-rbd-qos-docs
Josh Durgin [Thu, 13 Jan 2022 20:02:03 +0000 (12:02 -0800)]
Merge pull request #44554 from jdurgin/wip-rbd-qos-docs

doc/rbd: clarify and add more detail to librbd QoS docs

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #40802 from galsalomon66/wip-s3select-parquet-object-processing-2
Casey Bodley [Thu, 13 Jan 2022 17:53:33 +0000 (12:53 -0500)]
Merge pull request #40802 from galsalomon66/wip-s3select-parquet-object-processing-2

RGW/s3select : parquet implementation:

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44577 from clementperon/master
Kefu Chai [Thu, 13 Jan 2022 17:15:07 +0000 (01:15 +0800)]
Merge pull request #44577 from clementperon/master

cmake: Fix Finddpdk cmake module

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44498 from phlogistonjohn/jjm-root-check-later
Adam King [Thu, 13 Jan 2022 17:10:13 +0000 (12:10 -0500)]
Merge pull request #44498 from phlogistonjohn/jjm-root-check-later

cephadm: check if cephadm is root after cli is parsed

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #44394 from melissa-kun-li/enable-autotune
Adam King [Thu, 13 Jan 2022 17:06:46 +0000 (12:06 -0500)]
Merge pull request #44394 from melissa-kun-li/enable-autotune

Enable autotune for osd_memory_target on bootstrap

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
3 years agoMerge pull request #44306 from sebastian-philipp/normalize_image_digest-ambiguity
Adam King [Thu, 13 Jan 2022 17:03:50 +0000 (12:03 -0500)]
Merge pull request #44306 from sebastian-philipp/normalize_image_digest-ambiguity

cephadm: deal with ambiguity within normalize_image_digest

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
3 years agodoc/rbd/rbd-config-ref: add more detail on QoS settings 44554/head
Josh Durgin [Wed, 12 Jan 2022 03:17:15 +0000 (22:17 -0500)]
doc/rbd/rbd-config-ref: add more detail on QoS settings

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agohandling arm64(arrow installation) 40802/head
gal salomon [Thu, 13 Jan 2022 15:47:23 +0000 (17:47 +0200)]
handling arm64(arrow installation)

Signed-off-by: gal salomon <gal.salomon@gmail.com>
3 years agoMerge pull request #44427 from lxbsz/client_cleanup
Venky Shankar [Thu, 13 Jan 2022 15:04:54 +0000 (20:34 +0530)]
Merge pull request #44427 from lxbsz/client_cleanup

client: remove useless Lx cap check

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44451 from lxbsz/wip-53750
Venky Shankar [Thu, 13 Jan 2022 15:03:58 +0000 (20:33 +0530)]
Merge pull request #44451 from lxbsz/wip-53750

mds: directly return just after responding the link request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44561 from cbodley/wip-51727
Casey Bodley [Thu, 13 Jan 2022 14:38:49 +0000 (09:38 -0500)]
Merge pull request #44561 from cbodley/wip-51727

qa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #44538 from dang/wip-dang-zipper-perf
Daniel Gryniewicz [Thu, 13 Jan 2022 14:09:33 +0000 (09:09 -0500)]
Merge pull request #44538 from dang/wip-dang-zipper-perf

RGW Zipper - don't load stats for every bucket load

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44002 from JoshSalomon/wip-primary-balancer
Laura Flores [Thu, 13 Jan 2022 13:45:54 +0000 (07:45 -0600)]
Merge pull request #44002 from JoshSalomon/wip-primary-balancer

3 years agocmake: dpdk: only append common dir if it has been found 44577/head
Clément Péron [Thu, 13 Jan 2022 13:32:20 +0000 (14:32 +0100)]
cmake: dpdk: only append common dir if it has been found

Signed-off-by: Clément Péron <peron.clem@gmail.com>
3 years agocmake: dpdk: use STREQUAL and not EQUAL when comparing strings
Clément Péron [Thu, 13 Jan 2022 13:27:33 +0000 (14:27 +0100)]
cmake: dpdk: use STREQUAL and not EQUAL when comparing strings

Signed-off-by: Clément Péron <peron.clem@gmail.com>
3 years agocmake: dpdk: fix typo in HINTS when looking for DPDK
Clément Péron [Thu, 13 Jan 2022 13:26:29 +0000 (14:26 +0100)]
cmake: dpdk: fix typo in HINTS when looking for DPDK

Signed-off-by: Clément Péron <peron.clem@gmail.com>
3 years agomds: directly return just after responding the link request 44451/head
Xiubo Li [Tue, 4 Jan 2022 03:18:53 +0000 (11:18 +0800)]
mds: directly return just after responding the link request

Fixes: https://tracker.ceph.com/issues/53750
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #43286 from lxbsz/improve_setattr
Venky Shankar [Thu, 13 Jan 2022 12:53:27 +0000 (18:23 +0530)]
Merge pull request #43286 from lxbsz/improve_setattr

client: buffer the truncate if we have the Fx caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoclient: remove useless Lx cap check 44427/head
Xiubo Li [Thu, 30 Dec 2021 07:03:35 +0000 (15:03 +0800)]
client: remove useless Lx cap check

Once here the new_caps must have the 'Ls' caps, the extra check
for 'Lsx' makes no sense.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #44229 from lxbsz/mds-buffix
Venky Shankar [Thu, 13 Jan 2022 12:46:13 +0000 (18:16 +0530)]
Merge pull request #44229 from lxbsz/mds-buffix

mds: remove the duplicated or incorrect respond

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44397 from lxbsz/wip-53726
Venky Shankar [Thu, 13 Jan 2022 12:45:24 +0000 (18:15 +0530)]
Merge pull request #44397 from lxbsz/wip-53726

mds: dump tree '/' when the path is empty

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #44422 from lxbsz/wip-51705
Venky Shankar [Thu, 13 Jan 2022 12:44:14 +0000 (18:14 +0530)]
Merge pull request #44422 from lxbsz/wip-51705

qa: do not use any time related suffix for *_op_timeouts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43995 from TRYTOBE8TME/wip-rgw-kafka-teuth-cleanup
Yuval Lifshitz [Thu, 13 Jan 2022 09:57:03 +0000 (11:57 +0200)]
Merge pull request #43995 from TRYTOBE8TME/wip-rgw-kafka-teuth-cleanup

qa/tasks: Checking for kafka cleanup

3 years agoosd, tools: refactor OSDMap::calc_pg_upmaps (simplify the code) 44002/head
Josh Salomon [Thu, 13 Jan 2022 02:23:07 +0000 (02:23 +0000)]
osd, tools: refactor OSDMap::calc_pg_upmaps (simplify the code)

This is the first commit in a series of commits that aims at adding a primary balancer to Ceph and improving the current upmap balancer functionality. This first commit focuses on simplifying (refactoring) the code of `calc_pg_upmaps` so it is easier to change in the future. This PR keeps the existing functionality as-is and does not change anything but the code structure.

As part of the work is major refactoring of OSDMap::calc_pg_upmaps, the first thing is adding an --upmap-seed param to osdmaptool so test results can be compared without the random factor.

Other changes made:
    - Divided sections of `OSDMap::calc_pg_upmaps` into their own separate functions
    - Renamed tmp to tmp_osd_map
    - Changed all the occurances of 'first' and 'second' in the function to more meaningful names.

Signed-off-by: Josh Salomon <josh.salomon@gmail.com>
3 years agoMerge pull request #43299 from markhpc/wip-age-binning-rebase-20210923
Yuri Weinstein [Thu, 13 Jan 2022 00:54:23 +0000 (16:54 -0800)]
Merge pull request #43299 from markhpc/wip-age-binning-rebase-20210923

common/PriorityCache: Updated Implementation of Cache Age Binning

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoparquet implementation:
gal salomon [Mon, 12 Apr 2021 05:54:37 +0000 (08:54 +0300)]
parquet implementation:
(1) adding arrow/parquet to make(install is missing)
(2) s3select-operation contains 2 flows CSV and Parquet
(3) upon parquet-flow s3select processing engine is calling (via callback) to get-size and range-request, the range-requests are a-sync, thus the caller is waiting until notification.
(4) flow : execute --> s3select --(arrow layer)--> range-request --> GetObj::execute --> send_response_data --> notify-range-request --> (back-to) --> s3select
(5) on parquet flow the s3select is handling the response (using call-backs) because of aws-response-limitation (16mb)

add unique pointer (rgw_api); verify magic number for parquet objects; s3select module update
fix buffer-over-flow (copy range request)
change the range-request flow. now,it needs to use the callback parametrs (ofs & len) and not to use the element length
refactoring.  seperate the CSV flow from the parquet flow, a phase before adding conditional build(depend on arrow package installation)
adding arrow/parquet installation to debian/control
align s3select repo with RGW (missing API"s, such as get_error_description)
undefined reference to arrow symbol
fix comment: using optional_yield by value
fix comments; remove future/promise
s3select: a leak fix
s3select: fixing result production
s3select,s3tests : parquet alignments
typo: git-remote --> git_remote
s3select: remove redundant comma(end of projections); bug fix in parquet flow upon aggregation queries
adding arrow/parquet
editorial. remove blank lines
s3select: merged with master(output serialization,presto alignments)
merging(not rebase) master functionlities into parquet branch

(*) a dedicated source-files for s3select operation.
(*) s3select-engine: fix leaks on parquet flows, enabling allocate csv_object and parquet_object on stack
(*) the csv_object and parquet object allocated on stack (no heap allocation)

move data-members from heap to stack allocation, refactoring, separate flows for CSV and parquet. s3select: bug fix

conditional build: upon arrow package is installed the parquet flow become visable, thus enables to process parquet object. in case the package is not installed only CSV is usable

remove redundant try/catch, s3select: fix compile warning

arrow-devel version should be higher than 4.0.0, where arrow::io::AsyncContext become depecrated

missing sudo; wrong url;move the rm -f arrow.list

replace codename with $(lsb_release -sc)

arrow version should be >= 4.0.0; iocontext not exists in namespace on lower versions

RGW points to s3select/master

s3select submodule

sudo --> $SUDO

Signed-off-by: gal salomon <gal.salomon@gmail.com>
3 years agoqa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist 44561/head
Casey Bodley [Wed, 12 Jan 2022 19:07:26 +0000 (14:07 -0500)]
qa/rgw: add PG_DEGRADED cluster warnings to log-ignorelist

and cover rgw/singleton suite

Fixes: https://tracker.ceph.com/issues/51727
Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43494 from majianpeng/enable-test-librbd-BlockGuard
Ilya Dryomov [Wed, 12 Jan 2022 20:50:00 +0000 (21:50 +0100)]
Merge pull request #43494 from majianpeng/enable-test-librbd-BlockGuard

test/librbd: re-enable BlockGuard test

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #44478 from cyx1231st/wip-crimson-improve-log-3
Samuel Just [Wed, 12 Jan 2022 20:27:14 +0000 (12:27 -0800)]
Merge pull request #44478 from cyx1231st/wip-crimson-improve-log-3

crimson/os/seastore/../segment_manager: improve logs and validations

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #44536 from yuvalif/wip-yuval-dynamic-reshard
Casey Bodley [Wed, 12 Jan 2022 19:14:41 +0000 (14:14 -0500)]
Merge pull request #44536 from yuvalif/wip-yuval-dynamic-reshard

rgw: fix dynamic reshard happening during user stats sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoRGW Zipper - don't load stats for every bucket load 44538/head
Daniel Gryniewicz [Tue, 11 Jan 2022 15:50:00 +0000 (10:50 -0500)]
RGW Zipper - don't load stats for every bucket load

This was a side-effect of consolidating the Zipper API, and resulted in
a large performance hit.  Stats are only needed if they are requested,
so don't load them every time.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #39440 from pdvian/wip-warn-filestore-osds
Yuri Weinstein [Wed, 12 Jan 2022 16:49:02 +0000 (08:49 -0800)]
Merge pull request #39440 from pdvian/wip-warn-filestore-osds

mon/OSDMonitor, osd: Add warning on filestore deprecation and force use of wpq scheduler for filestore OSDs

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #41778 from felixhuettner/fix_subdir_name
Daniel Gryniewicz [Wed, 12 Jan 2022 14:22:04 +0000 (09:22 -0500)]
Merge pull request #41778 from felixhuettner/fix_subdir_name

rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

3 years agoMerge pull request #38234 from inspur-wyq/wip-copy-obj-check-size
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:48 +0000 (09:21 -0500)]
Merge pull request #38234 from inspur-wyq/wip-copy-obj-check-size

rgw : check the object size when copy obj

3 years agoMerge pull request #38532 from Rjerk/wip-empty-tagset
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:35 +0000 (09:21 -0500)]
Merge pull request #38532 from Rjerk/wip-empty-tagset

rgw: an empty tagset is allowed by S3

3 years agoMerge pull request #40573 from Huber-ming/rgw_admin
Daniel Gryniewicz [Wed, 12 Jan 2022 14:21:15 +0000 (09:21 -0500)]
Merge pull request #40573 from Huber-ming/rgw_admin

rgw: delete abbreviation for option "--new-uid"

3 years agoMerge pull request #40575 from Huber-ming/rgw_admin-f
Daniel Gryniewicz [Wed, 12 Jan 2022 14:20:49 +0000 (09:20 -0500)]
Merge pull request #40575 from Huber-ming/rgw_admin-f

radosgw-admin: delete the abbreviation of option "--infile"

3 years agoMerge pull request #44500 from idryomov/wip-rbd-test-group-leak
Ilya Dryomov [Wed, 12 Jan 2022 11:56:09 +0000 (12:56 +0100)]
Merge pull request #44500 from idryomov/wip-rbd-test-group-leak

test/librbd: fix group_info.name leaks in TestGroup.add_image

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44490 from liu-chunmei/crimson-fix-aligned
Liu-Chunmei [Wed, 12 Jan 2022 06:05:22 +0000 (22:05 -0800)]
Merge pull request #44490 from liu-chunmei/crimson-fix-aligned

crimson: fix assert_aligned(size) in trim_data_reservation

reviewed by: Samuel Just <sjust@redhat.com> , Yingxin <yingxin.cheng@intel.com>

3 years agocrimson/os/seastore/journal: fast submit if RecordSubmitter is IDLE and no pending 44555/head
Yingxin Cheng [Wed, 12 Jan 2022 05:35:47 +0000 (13:35 +0800)]
crimson/os/seastore/journal: fast submit if RecordSubmitter is IDLE and no pending

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: add more validations 44478/head
Yingxin Cheng [Thu, 6 Jan 2022 08:03:20 +0000 (16:03 +0800)]
crimson/os/seastore/../segment_manager: add more validations

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: consolidate logs with structured level and...
Yingxin Cheng [Thu, 6 Jan 2022 08:01:28 +0000 (16:01 +0800)]
crimson/os/seastore/../segment_manager: consolidate logs with structured level and format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: cleanup device_id usage
Yingxin Cheng [Thu, 6 Jan 2022 02:26:20 +0000 (10:26 +0800)]
crimson/os/seastore/../segment_manager: cleanup device_id usage

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: pretty print data structures
Yingxin Cheng [Wed, 5 Jan 2022 14:56:13 +0000 (22:56 +0800)]
crimson/os/seastore/../segment_manager: pretty print data structures

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agodoc/rbd/rbd-config-ref: group QoS options by throttle type
Josh Durgin [Wed, 12 Jan 2022 02:44:05 +0000 (21:44 -0500)]
doc/rbd/rbd-config-ref: group QoS options by throttle type

This makes it clearer that there are distinct throttles with the
same groups of settings.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agocrimson/os/seastore/../segment_manager: convert to seastore logging
Yingxin Cheng [Wed, 5 Jan 2022 05:32:46 +0000 (13:32 +0800)]
crimson/os/seastore/../segment_manager: convert to seastore logging

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agocrimson/os/seastore/../segment_manager: suppress compile warning about unused logger
Yingxin Cheng [Wed, 5 Jan 2022 03:22:09 +0000 (11:22 +0800)]
crimson/os/seastore/../segment_manager: suppress compile warning about unused logger

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #44532 from rzarzynski/wip-crimson-fix-test-runner
Yingxin [Wed, 12 Jan 2022 01:23:53 +0000 (09:23 +0800)]
Merge pull request #44532 from rzarzynski/wip-crimson-fix-test-runner

test/crimson: fix a race condition in SeastarRunner

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #43593 from ljflores/wip-rocksdb
Neha Ojha [Wed, 12 Jan 2022 00:31:41 +0000 (16:31 -0800)]
Merge pull request #43593 from ljflores/wip-rocksdb

mgr: expose rocksdb version number for use in telemetry

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agocrimson: fix assert_aligned(size) in trim_data_reservation 44490/head
chunmei-liu [Tue, 11 Jan 2022 03:44:43 +0000 (19:44 -0800)]
crimson: fix assert_aligned(size) in trim_data_reservation

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
signed-off-by: Samuel Just <sjust@redhat.com>

3 years agomgr/telemetry: add the rocksdb version number to telemetry 43593/head
Laura Flores [Tue, 4 Jan 2022 22:54:33 +0000 (22:54 +0000)]
mgr/telemetry: add the rocksdb version number to telemetry

Capturing the RocksDB version number in Telemetry would allow us to check that users are using the appropriate RocksDB version for their Ceph cluster. For instance, if a user is working in a Pacific cluster, but their RocksDB version is meant for Nautilus, that might be a problem.

It is strucured as "rocksdb_stats" --> "version" in anticipation of more stats that can will be added under "rocksdb_stats".

Signed-off-by: Laura Flores <lflores@redhat.com>
3 years agomgr: expose rocksdb version number in the mgr module
Laura Flores [Wed, 22 Dec 2021 23:09:25 +0000 (23:09 +0000)]
mgr: expose rocksdb version number in the mgr module

It is only necessary here to link the rocksdb include directory
since the mgr simply needs access to the rocksdb version numbers.

Signed-off-by: Laura Flores <lflores@redhat.com>
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Co-authored-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #43794 from aclamk/wip-bluefs-fine-grain-locking-4
Neha Ojha [Tue, 11 Jan 2022 21:59:16 +0000 (13:59 -0800)]
Merge pull request #43794 from aclamk/wip-bluefs-fine-grain-locking-4

os/bluestore: BlueFS fine grain locking

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agocommon/PriorityCache: Implement Cache Age Binning 43299/head
Mark Nelson [Tue, 13 Oct 2020 23:24:18 +0000 (23:24 +0000)]
common/PriorityCache: Implement Cache Age Binning

Signed-off-by: Mark Nelson <mnelson@redhat.com>
3 years agoMerge pull request #44537 from pdvian/wip-typo-fix
Casey Bodley [Tue, 11 Jan 2022 20:50:34 +0000 (15:50 -0500)]
Merge pull request #44537 from pdvian/wip-typo-fix

rgw: Fix typo error for stat_remote_obj

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #44496 from idryomov/wip-diff-iterate-parent-fix
Ilya Dryomov [Tue, 11 Jan 2022 19:28:19 +0000 (20:28 +0100)]
Merge pull request #44496 from idryomov/wip-diff-iterate-parent-fix

librbd: restore diff-iterate include_parent functionality in fast-diff mode

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agorgw: Fix typo error for stat_remote_obj 44537/head
Prashant D [Tue, 11 Jan 2022 18:58:20 +0000 (13:58 -0500)]
rgw: Fix typo error for stat_remote_obj

Signed-off-by: Prashant D <pdhange@redhat.com>
3 years agorgw: fix dynamic reshard happening during user stats sync 44536/head
Yuval Lifshitz [Mon, 10 Jan 2022 16:55:50 +0000 (18:55 +0200)]
rgw: fix dynamic reshard happening during user stats sync

this is a regression caused by the following commit:
https://github.com/ceph/ceph/commit/72d1a363263cf707d022ee756122236ba175cda2

to verify use the following instructions:
https://gist.github.com/yuvalif/d526c0a3a4c5b245b9e951a6c5a10517

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #44483 from idryomov/wip-diff-iterate-offset-fix
Ilya Dryomov [Tue, 11 Jan 2022 17:59:12 +0000 (18:59 +0100)]
Merge pull request #44483 from idryomov/wip-diff-iterate-offset-fix

librbd: diff-iterate reports incorrect offsets in fast-diff mode

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44355 from sseshasa/wip-fix-enhance-mclock-logging
Yuri Weinstein [Tue, 11 Jan 2022 17:32:12 +0000 (09:32 -0800)]
Merge pull request #44355 from sseshasa/wip-fix-enhance-mclock-logging

osd:  Enhance mclockscheduler logging and formatted dump information.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44482 from sebastian-philipp/nfs-upgrade-port-conflict
Yuri Weinstein [Tue, 11 Jan 2022 17:31:35 +0000 (09:31 -0800)]
Merge pull request #44482 from sebastian-philipp/nfs-upgrade-port-conflict

mgr/cephadm: nfs migration: avoid port conflicts

Reviewed-by: Adam King adking@redhat.com
3 years agoMerge pull request #36308 from linuxbox2/wip-rgwlc-bcktdel
Matt Benjamin [Tue, 11 Jan 2022 17:17:22 +0000 (12:17 -0500)]
Merge pull request #36308 from linuxbox2/wip-rgwlc-bcktdel

rgwlc:  remove lc entry on bucket delete

3 years agoMerge pull request #42891 from ofriedma/wip-ofriedma-rgw-qos-finale
Casey Bodley [Tue, 11 Jan 2022 16:35:05 +0000 (11:35 -0500)]
Merge pull request #42891 from ofriedma/wip-ofriedma-rgw-qos-finale

rgw: Add rgw rate limiting per user and per bucket

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #38774 from linuxbox2/wip-doc-replsupport
Casey Bodley [Tue, 11 Jan 2022 16:13:24 +0000 (11:13 -0500)]
Merge pull request #38774 from linuxbox2/wip-doc-replsupport

rgw: document S3 bucket replication support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoqa/tasks: Checking for kafka cleanup 43995/head
Kalpesh Pandya [Thu, 11 Nov 2021 06:46:16 +0000 (12:16 +0530)]
qa/tasks: Checking for kafka cleanup

Adding a sleep after running ./kafka-server-stop.sh and ./zookeeper-server-stop.sh
scripts so that nothing gets logged into the kafka logs after the sleep time.
And finally killing the process.

This resolves: https://tracker.ceph.com/issues/53220

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
3 years agotest/crimson: fix a race condition in SeastarRunner 44532/head
Radoslaw Zarzynski [Tue, 11 Jan 2022 12:09:28 +0000 (12:09 +0000)]
test/crimson: fix a race condition in SeastarRunner

This patch is supposed to fix the following problem:

```
        Start 234: unittest-seastar-errorator
216/258 Test #234: unittest-seastar-errorator ................Child aborted***Exception:   0.95 sec
WARNING: debug mode. Not for benchmarking or production
WARN  2022-01-11 10:01:32,973 [shard 0] seastar - Creation of perf_event based stall detector failed, falling back to posix timer: std::system_error (error system:13, perf_event_open() failed: Permission denied)
unittest-seastar-errorator: ../src/test/crimson/seastar_runner.h:45: int SeastarRunner::init(int, char **): Assertion `begin_signaled == true' failed.
Aborting.
Backtrace:
  0xb3397a
  0x1d19430
  0x1d1905d
  0x1b72332
  0x1b9dd95
  0x1c75fe9
  0x1c76231
  0x1c7605a
  0x7f20b7aee3bf
  /lib/x86_64-linux-gnu/libc.so.6+0x4618a
  /lib/x86_64-linux-gnu/libc.so.6+0x25858
  /lib/x86_64-linux-gnu/libc.so.6+0x25728
  /lib/x86_64-linux-gnu/libc.so.6+0x36f35
  0xc8bb88
  0xc7ec71
  /lib/x86_64-linux-gnu/libc.so.6+0x270b2
  0xafc95d
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #43404 from rhcs-dashboard/angular-12-upgradee
Ernesto Puerta [Tue, 11 Jan 2022 12:02:07 +0000 (13:02 +0100)]
Merge pull request #43404 from rhcs-dashboard/angular-12-upgradee

mgr/dashboard: Update Angular version to 12

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
3 years agoMerge pull request #44088 from ceph/feature-48388-cache
Ernesto Puerta [Tue, 11 Jan 2022 11:52:41 +0000 (12:52 +0100)]
Merge pull request #44088 from ceph/feature-48388-cache

mgr: TTL cache implementation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
3 years agoMerge pull request #44294 from rhcs-dashboard/feature-bluestore-onode
Ernesto Puerta [Tue, 11 Jan 2022 10:24:21 +0000 (11:24 +0100)]
Merge pull request #44294 from rhcs-dashboard/feature-bluestore-onode

mgr/dashboard: monitoring:Implement BlueStore onode hit/miss counters into the dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44456 from jtlayton/wip-53765
Venky Shankar [Tue, 11 Jan 2022 09:09:16 +0000 (14:39 +0530)]
Merge pull request #44456 from jtlayton/wip-53765

mount.ceph: fix the handling of new-syntax device names

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoosd: Display scheduler specific info when dumping an OpSchedulerItem 44355/head
Sridhar Seshasayee [Fri, 17 Dec 2021 12:30:05 +0000 (18:00 +0530)]
osd: Display scheduler specific info when dumping an OpSchedulerItem

Implement logic to dump information relevant to the scheduler type being
employed when dumping details about an OpSchedulerItem. For e.g., the
'priority' field is relevant for the 'wpq' scheduler, but for the
'mclock_scheduler', the 'qos_cost' gives more information during debugging.

A couple of additional fields called 'qos_cost' and 'is_qos_request' are
introduced in OpSchedulerItem class. These are mainly used to facilitate
dumping of relevant information depending on the scheduler type. The
interesting points are when an item is enqueued and dequeued.

For the 'mclock_scheduler', the 'class_id' and the 'qos_cost' fields are
dumped during enqueue and dequeue op respectively. For the 'wpq' scheduler
things remain the same as before.

An additional benefit of this change is to help immediately identify the
type of scheduler being used for a given shard depending on what is dumped
in the debug messages while debugging.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoosd: Add debug logs and formatted dumps in the mClockScheduler
Sridhar Seshasayee [Tue, 14 Dec 2021 10:41:14 +0000 (16:11 +0530)]
osd: Add debug logs and formatted dumps in the mClockScheduler

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoosd: Change description of 'dump_op_pq_state' command to be more generic
Sridhar Seshasayee [Thu, 6 Jan 2022 09:58:21 +0000 (15:28 +0530)]
osd: Change description of 'dump_op_pq_state' command to be more generic

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agocommon: Introduce 'mclock' debug subsystem
Sridhar Seshasayee [Thu, 6 Jan 2022 09:56:20 +0000 (15:26 +0530)]
common: Introduce 'mclock' debug subsystem

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
3 years agoMerge pull request #44410 from tchaikov/wip-cmake-cleanup
Samuel Just [Mon, 10 Jan 2022 21:36:09 +0000 (13:36 -0800)]
Merge pull request #44410 from tchaikov/wip-cmake-cleanup

cmake: detect linux/blk/zoned support

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #44502 from yuvalif/fix-call-to-mrgw
Yuval Lifshitz [Mon, 10 Jan 2022 20:05:22 +0000 (22:05 +0200)]
Merge pull request #44502 from yuvalif/fix-call-to-mrgw

rgw/multisite/test: pass zero (no) ssl port to mrgw.sh

3 years agoMerge pull request #44495 from Matan-B/wip-matanb-crimson-flavor
Matan [Mon, 10 Jan 2022 18:30:18 +0000 (20:30 +0200)]
Merge pull request #44495 from Matan-B/wip-matanb-crimson-flavor

qa/suites: Crimson flavor configuration fix

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: chunmei-liu chunmei.liu@intel.com
3 years agotest/librbd: make diff-iterate clone tests exercise fast-diff mode 44496/head
Ilya Dryomov [Fri, 7 Jan 2022 12:31:08 +0000 (13:31 +0100)]
test/librbd: make diff-iterate clone tests exercise fast-diff mode

The fast-diff feature wasn't propagated to the clone so these tests
were exercising the slow list_snaps path no matter what RBD_FEATURES
value was supplied to ceph_test_librbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: restore diff-iterate include_parent functionality in fast-diff mode
Ilya Dryomov [Wed, 5 Jan 2022 19:24:40 +0000 (20:24 +0100)]
librbd: restore diff-iterate include_parent functionality in fast-diff mode

Commit 4429ed4f3f4c ("librbd: switch diff iterate API to use new snaps
list dispatch methods") removed the recursive execute() call.  The new
list_snaps method does indeed handle parent diffs internally but it is
not used in fast-diff mode.  Nothing changed there -- we still need to
load the parent object map, calculate parent object_diff_state, etc.

Fixes: https://tracker.ceph.com/issues/53787
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agolibrbd: stash unmodified include_parent value in DiffContext
Ilya Dryomov [Wed, 5 Jan 2022 18:45:50 +0000 (19:45 +0100)]
librbd: stash unmodified include_parent value in DiffContext

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agotest/librbd: fix group_info.name leaks in TestGroup.add_image 44500/head
Ilya Dryomov [Sat, 8 Jan 2022 13:18:32 +0000 (14:18 +0100)]
test/librbd: fix group_info.name leaks in TestGroup.add_image

Introduced in commit c04e8c3374e0 ("test/librbd: add get_group test").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
3 years agodoc: rgw: document S3 bucket replication support 38774/head
Matt Benjamin [Tue, 5 Jan 2021 20:30:23 +0000 (15:30 -0500)]
doc: rgw: document S3 bucket replication support

Support was added at Octopus.

Fixes: https://tracker.ceph.com/issues/48755
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
3 years agorgw: Add admin ops API for rate limiting 42891/head
Or Friedmann [Mon, 3 Jan 2022 16:32:16 +0000 (16:32 +0000)]
rgw: Add admin ops API for rate limiting

Add admin ops API for rate limiting and some bug fixes

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
3 years agoMerge pull request #44221 from BenoitKnecht/ceph-mon-device-metrics
Yuri Weinstein [Mon, 10 Jan 2022 15:56:29 +0000 (07:56 -0800)]
Merge pull request #44221 from BenoitKnecht/ceph-mon-device-metrics

mon: Abort device health when device not found

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
3 years agoMerge pull request #43362 from ybwang0211/pubsub_settingng_spell
Daniel Gryniewicz [Mon, 10 Jan 2022 15:10:51 +0000 (10:10 -0500)]
Merge pull request #43362 from ybwang0211/pubsub_settingng_spell

rgw:word spelling error in rgw/rgw_pubsub.h file

3 years agoMerge pull request #42115 from sandrobonazzola/patch-2
Sebastian Wagner [Mon, 10 Jan 2022 14:41:32 +0000 (15:41 +0100)]
Merge pull request #42115 from sandrobonazzola/patch-2

doc/install: ensure to open ceph-mon firewall ports

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agorgw/multisite/test: passzero (no) ssl port to mrgw.sh 44502/head
Yuval Lifshitz [Sun, 9 Jan 2022 16:27:40 +0000 (18:27 +0200)]
rgw/multisite/test: passzero (no) ssl port to mrgw.sh

in multisite tests script.
this is to fix an issue from: 1f3fbf218faa4859f02917828dbdb48be41a579a

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
3 years agoMerge pull request #44301 from yuvalif/wip-yuval-cloudevents
Yuval Lifshitz [Sun, 9 Jan 2022 07:32:53 +0000 (09:32 +0200)]
Merge pull request #44301 from yuvalif/wip-yuval-cloudevents

rgw/notifications: add cloudevents support to HTTP endpoint

3 years agoMerge pull request #44278 from fengchunsong/dpdk-tso
Kefu Chai [Sat, 8 Jan 2022 11:28:00 +0000 (19:28 +0800)]
Merge pull request #44278 from fengchunsong/dpdk-tso

msg/async/dpdk: Add an option to enable/disable TSO

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44198 from wjwithagen/fix-bluestore-get-io-size
Kefu Chai [Sat, 8 Jan 2022 04:29:21 +0000 (12:29 +0800)]
Merge pull request #44198 from wjwithagen/fix-bluestore-get-io-size

common/blkdev: add missing stubs for OSes not Linux

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #44402 from inspur-wyq/wip-pybind-mgr
Yuri Weinstein [Fri, 7 Jan 2022 23:36:58 +0000 (15:36 -0800)]
Merge pull request #44402 from inspur-wyq/wip-pybind-mgr

pybind/mgr: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #44055 from ifed01/wip-ifed-cosmetics-for-optimize-rm
Yuri Weinstein [Fri, 7 Jan 2022 23:36:37 +0000 (15:36 -0800)]
Merge pull request #44055 from ifed01/wip-ifed-cosmetics-for-optimize-rm

test/store_test, os/bluestore: bunch of minor improvements/cleanups

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup
Yuri Weinstein [Fri, 7 Jan 2022 23:35:42 +0000 (15:35 -0800)]
Merge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup

cmake: cleanups about systemd and install

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #44423 from 5cs/fix-unshare-blob-test
Igor Fedotov [Fri, 7 Jan 2022 23:27:42 +0000 (02:27 +0300)]
Merge pull request #44423 from 5cs/fix-unshare-blob-test

test: fix bluestore unshare blob test case

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
3 years agoMerge pull request #44416 from yuvalif/wip-yuval-rgw-params-in-multisite
Casey Bodley [Fri, 7 Jan 2022 20:19:44 +0000 (15:19 -0500)]
Merge pull request #44416 from yuvalif/wip-yuval-rgw-params-in-multisite

rgw/multisite/test: allow passing rgw parameter to multisite script

Reviewed-by: Casey Bodley <cbodley@redhat.com>