]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agomds: print is_laggy message once 24138/head
Patrick Donnelly [Sun, 2 Sep 2018 01:47:18 +0000 (18:47 -0700)]
mds: print is_laggy message once

Beacon::is_laggy gets called frequently which causes the debug log to get
spammed with messages.

Steps to reproduce:
- setup vstart cluster
- start IO from client (e.g. `cp -av /usr /mnt`)
- find connection to mon by the mds
  $ ss -tnp | grep ceph-mds
    ...
    ESTAB      0      0      127.0.0.1:58988              127.0.0.1:40928               users:(("ceph-mds",pid=13095,fd=37))
- block the connection using iptables on the MDS node:
  $ sudo iptables -I INPUT -p tcp --sport 58988 --dport 40928 -j DROP
- verify MDS log using:
  $ tail -f mds.a.log | grep beacon
    ...
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.784 7f2905a97700  5 mds.beacon.a Sending beacon up:active seq 114
    2018-09-05 19:21:05.784 7f2905a97700  1 -- 127.0.0.1:6813/2277776624 --> 127.0.0.1:40929/0 -- mdsbeacon(14162/a up:active seq 114 v7) v7 -- 0x7d1e7e2a80 con 0
    2018-09-05 19:21:05.784 7f2905a97700 20 mds.beacon.a sender thread waiting interval 4s

Fixes: http://tracker.ceph.com/issues/35250
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 439902311910f9c239f48c0958fd95fa5839e309)

Conflicts:
src/mds/Beacon.cc: Resovled for laggy, changed to was_laggy

6 years agoMerge pull request #23408 from ukernel/luminous-24820
Yuri Weinstein [Mon, 17 Sep 2018 19:48:11 +0000 (12:48 -0700)]
Merge pull request #23408 from ukernel/luminous-24820

luminous: mds: avoid using g_conf->get_val<...>(...) in hot path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23505 from ukernel/luminous-26834
Yuri Weinstein [Mon, 17 Sep 2018 19:47:39 +0000 (12:47 -0700)]
Merge pull request #23505 from ukernel/luminous-26834

luminous: mds: calculate load by checking self CPU usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23507 from ukernel/luminous-26858
Yuri Weinstein [Mon, 17 Sep 2018 19:47:10 +0000 (12:47 -0700)]
Merge pull request #23507 from ukernel/luminous-26858

luminous: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23554 from batrick/i26906
Yuri Weinstein [Mon, 17 Sep 2018 19:46:44 +0000 (12:46 -0700)]
Merge pull request #23554 from batrick/i26906

luminous: MDSMonitor: consider raising priority of MMDSBeacons from MDS so they are processed before other client messages

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #23556 from batrick/i26924
Yuri Weinstein [Mon, 17 Sep 2018 19:46:09 +0000 (12:46 -0700)]
Merge pull request #23556 from batrick/i26924

luminous: mds: mds got laggy because of MDSBeacon stuck in mqueue

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #23617 from pdvian/wip-26915-luminous
Yuri Weinstein [Mon, 17 Sep 2018 19:45:37 +0000 (12:45 -0700)]
Merge pull request #23617 from pdvian/wip-26915-luminous

luminous: client: check for unmounted condition before printing debug output

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23677 from ukernel/luminous-26894
Yuri Weinstein [Mon, 17 Sep 2018 19:44:48 +0000 (12:44 -0700)]
Merge pull request #23677 from ukernel/luminous-26894

luminous: mds: crash when dumping ops in flight

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23678 from ukernel/luminous-24840
Yuri Weinstein [Mon, 17 Sep 2018 19:44:23 +0000 (12:44 -0700)]
Merge pull request #23678 from ukernel/luminous-24840

luminous: mds: explain delayed client_request due to subtree migration

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23693 from tserlin/tserlin-require-fuse-luminous
Yuri Weinstein [Mon, 17 Sep 2018 19:44:00 +0000 (12:44 -0700)]
Merge pull request #23693 from tserlin/tserlin-require-fuse-luminous

luminous: debian/control: require fuse for ceph-fuse

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #23726 from joscollin/wip-25205-luminous
Yuri Weinstein [Mon, 17 Sep 2018 19:43:06 +0000 (12:43 -0700)]
Merge pull request #23726 from joscollin/wip-25205-luminous

luminous: CephVolumeClient: delay required after adding data pool to MDSMap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23758 from dillaman/wip-27987
Yuri Weinstein [Mon, 17 Sep 2018 19:29:13 +0000 (12:29 -0700)]
Merge pull request #23758 from dillaman/wip-27987

luminous: librbd: fix refuse to release lock when cookie is the same at rewatch

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #23892 from pdvian/wip-27061-luminous
Yuri Weinstein [Mon, 17 Sep 2018 19:28:26 +0000 (12:28 -0700)]
Merge pull request #23892 from pdvian/wip-27061-luminous

luminous: librbd: potential race on image create request complete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24003 from pdvian/wip-35704-luminous
Yuri Weinstein [Mon, 17 Sep 2018 19:27:51 +0000 (12:27 -0700)]
Merge pull request #24003 from pdvian/wip-35704-luminous

luminous: rbd: fix error import when the input is a pipe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #23680 from k0ste/luminous_backports
Yuri Weinstein [Fri, 14 Sep 2018 16:47:36 +0000 (09:47 -0700)]
Merge pull request #23680 from k0ste/luminous_backports

luminous: osd: backport 'bench' and stdout changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23740 from k0ste/luminous_backports2
Yuri Weinstein [Fri, 14 Sep 2018 16:46:50 +0000 (09:46 -0700)]
Merge pull request #23740 from k0ste/luminous_backports2

luminous: mgr/balancer: deepcopy best plan - otherwise we get latest

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23791 from tchaikov/wip-23370-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:46:18 +0000 (09:46 -0700)]
Merge pull request #23791 from tchaikov/wip-23370-luminous

luminous: mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23862 from smithfarm/wip-25177-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:45:41 +0000 (09:45 -0700)]
Merge pull request #23862 from smithfarm/wip-25177-luminous

luminous: osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #23864 from smithfarm/wip-25203-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:45:09 +0000 (09:45 -0700)]
Merge pull request #23864 from smithfarm/wip-25203-luminous

luminous: rados python bindings use prval from stack

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23871 from smithfarm/wip-32106-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:44:25 +0000 (09:44 -0700)]
Merge pull request #23871 from smithfarm/wip-32106-luminous

luminous: object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #23901 from pdvian/wip-32078-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:43:52 +0000 (09:43 -0700)]
Merge pull request #23901 from pdvian/wip-32078-luminous

luminous: test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #23943 from batrick/i35716
Yuri Weinstein [Fri, 14 Sep 2018 16:43:12 +0000 (09:43 -0700)]
Merge pull request #23943 from batrick/i35716

luminous: msg: "challenging authorizer" messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #23969 from zmedico/backport-lumious-17121-17239-17410-17820
Yuri Weinstein [Fri, 14 Sep 2018 16:42:50 +0000 (09:42 -0700)]
Merge pull request #23969 from zmedico/backport-lumious-17121-17239-17410-17820

luminous: PGPool::update optimizations

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23499 from pdvian/wip-26844-luminous
Yuri Weinstein [Fri, 14 Sep 2018 16:40:50 +0000 (09:40 -0700)]
Merge pull request #23499 from pdvian/wip-26844-luminous

luminous: rgw_file: deep stat handling

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #23522 from pdvian/wip-26846-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:41:51 +0000 (08:41 -0700)]
Merge pull request #23522 from pdvian/wip-26846-luminous

luminous: rgw: Limit the number of lifecycle rules on one bucket.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
6 years agoMerge pull request #23545 from pdvian/wip-26848-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:41:18 +0000 (08:41 -0700)]
Merge pull request #23545 from pdvian/wip-26848-luminous

luminous: rgw: The delete markers generated by object expiration should have owner

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
6 years agoMerge pull request #23665 from pdvian/wip-26922-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:40:43 +0000 (08:40 -0700)]
Merge pull request #23665 from pdvian/wip-26922-luminous

luminous: rgw: Fix log level of gc_iterate_entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23974 from pdvian/wip-35069-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:40:17 +0000 (08:40 -0700)]
Merge pull request #23974 from pdvian/wip-35069-luminous

cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24034 from smithfarm/wip-25087-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:39:12 +0000 (08:39 -0700)]
Merge pull request #24034 from smithfarm/wip-25087-luminous

luminous: change default rgw_thread_pool_size to 512

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24046 from pdvian/wip-35707-luminous
Yuri Weinstein [Fri, 14 Sep 2018 15:38:42 +0000 (08:38 -0700)]
Merge pull request #24046 from pdvian/wip-35707-luminous

luminous: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24082 from alfredodeza/luminous-rm35970
Andrew Schoen [Thu, 13 Sep 2018 18:01:28 +0000 (13:01 -0500)]
Merge pull request #24082 from alfredodeza/luminous-rm35970

luminous: ceph-volume fix zap not working with LVs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24080 from alfredodeza/luminous-rm35913
Alfredo Deza [Thu, 13 Sep 2018 17:59:27 +0000 (13:59 -0400)]
Merge pull request #24080 from alfredodeza/luminous-rm35913

luminous: ceph-volume batch: allow --osds-per-device, default it to 1

Reviewed-by: Andrew Schoen aschoen@redhat.com
6 years agoceph-volume tests.functional comment why not use --destroy when zapping lvs in filestore 24082/head
Alfredo Deza [Thu, 13 Sep 2018 14:03:06 +0000 (10:03 -0400)]
ceph-volume tests.functional comment why not use --destroy when zapping lvs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 11858c74a1e704d63185f0ef4b6cd6aa02f17e52)

6 years agoceph-volume tests.functional use --destroy when zapping lvs in bluestore
Alfredo Deza [Thu, 13 Sep 2018 14:00:37 +0000 (10:00 -0400)]
ceph-volume tests.functional use --destroy when zapping lvs in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 87af815691a7f43d14067c24e308d60f39d58051)

6 years agoceph-volume lvm.zap use remove_pv only for physical devices
Alfredo Deza [Thu, 13 Sep 2018 13:34:54 +0000 (09:34 -0400)]
ceph-volume lvm.zap use remove_pv only for physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 036683fde799ab1cfad7d6192981515d6b3419cc)

6 years agoceph-volume tests.lvm.strategies initial filestore validation 24080/head
Alfredo Deza [Wed, 12 Sep 2018 13:44:32 +0000 (09:44 -0400)]
ceph-volume tests.lvm.strategies initial filestore validation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d1d68eade56ba3f898cd3bfd29aa1ee3bf5460ec)

6 years agoceph-volume lvm.tests conftest fixtures for custom vgs
Alfredo Deza [Tue, 11 Sep 2018 20:53:54 +0000 (16:53 -0400)]
ceph-volume lvm.tests conftest fixtures for custom vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f14db432317c15b29b187d1e1a4bd3a729a7043)

6 years agoceph-volume tests.devices.lvm add bluestore strategy cases
Alfredo Deza [Tue, 11 Sep 2018 20:53:05 +0000 (16:53 -0400)]
ceph-volume tests.devices.lvm add bluestore strategy cases

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2ced05a2bc6ba01cf8946f43d5839890a28cab20)

6 years agoceph-volume tests verify strategy validators for minimum size
Alfredo Deza [Tue, 11 Sep 2018 16:57:21 +0000 (12:57 -0400)]
ceph-volume tests verify strategy validators for minimum size

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9167f76c2a7107c82b781cbe5f51e24ee72e4db4)

6 years agoceph-volume lvm.batch.filestore allow osds-per-device
Alfredo Deza [Tue, 11 Sep 2018 16:12:48 +0000 (12:12 -0400)]
ceph-volume lvm.batch.filestore allow osds-per-device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 66947537b542da82185ea4d79bfaedaae3117918)

6 years agoceph-volume lvm.batch.validators account for osds per device, include collocation
Alfredo Deza [Tue, 11 Sep 2018 16:11:01 +0000 (12:11 -0400)]
ceph-volume lvm.batch.validators account for osds per device, include collocation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04a40eda0ac089c07654d9ab07958126f0fc8f84)

6 years agoceph-volume lvm.batch.bluestore allow osds-per-device with MixedType
Alfredo Deza [Mon, 10 Sep 2018 21:57:44 +0000 (17:57 -0400)]
ceph-volume lvm.batch.bluestore allow osds-per-device with MixedType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 028ef60df768c24a9573a044a451164dcb037677)

6 years agoceph-volume lvm.batch.bluestore allow osds-per-device with SingleType
Alfredo Deza [Mon, 10 Sep 2018 21:01:57 +0000 (17:01 -0400)]
ceph-volume lvm.batch.bluestore allow osds-per-device with SingleType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8c0e707110e284626fe4161097a3121170a3f09d)

6 years agoceph-volume lvm.batch add --osds-per-device flag, defaulting to 1
Alfredo Deza [Mon, 10 Sep 2018 20:12:59 +0000 (16:12 -0400)]
ceph-volume lvm.batch add --osds-per-device flag, defaulting to 1

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe7c5e0129172a4b2a216e7c49e67d8ead570b39)

6 years agoMerge pull request #24075 from alfredodeza/luminous-rm34535
Andrew Schoen [Thu, 13 Sep 2018 12:51:31 +0000 (07:51 -0500)]
Merge pull request #24075 from alfredodeza/luminous-rm34535

luminous: ceph-volume batch carve out lvs for bluestore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.util verify Disk objects don't change state with divisions 24075/head
Alfredo Deza [Mon, 10 Sep 2018 16:28:57 +0000 (12:28 -0400)]
ceph-volume tests.util verify Disk objects don't change state with divisions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ca526336221ea2ad32e7daf3e5e46b07291a1d8)

6 years agoceph-volume util.disk fix an issue where Disk objects would mutate on div operations
Alfredo Deza [Mon, 10 Sep 2018 16:18:49 +0000 (12:18 -0400)]
ceph-volume util.disk fix an issue where Disk objects would mutate on div operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d1a3c74240e67621e993a304eaf566d4e2789d1)

6 years agoceph-volume util.prepare add a helper to get block.db sizes from ceph.conf
Alfredo Deza [Mon, 10 Sep 2018 16:09:50 +0000 (12:09 -0400)]
ceph-volume util.prepare add a helper to get block.db sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860068ec5daca6d7087cf1ce0594a2fe36e91193)

6 years agoceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices
Alfredo Deza [Mon, 10 Sep 2018 16:09:15 +0000 (12:09 -0400)]
ceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0eb0bdfc55d8ceb27cd12bc3a7fb807a7e65c788)

6 years ago-f ceph-volume lvm.batch.bluestore validation and reporting with VG reuse
Alfredo Deza [Mon, 10 Sep 2018 16:08:35 +0000 (12:08 -0400)]
-f  ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d117d66903fa740108c9948b499150b02523fdf4)

6 years agoceph-volume lvm.batch fix error reporting, Device objects aren't strings
Alfredo Deza [Fri, 7 Sep 2018 18:11:33 +0000 (14:11 -0400)]
ceph-volume lvm.batch fix error reporting, Device objects aren't strings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 267a386cc270081b2cd284bf78bf94b52be113dc)

6 years agoceph-volume lvm.batch.bluestore validation and reporting with VG reuse
Alfredo Deza [Fri, 7 Sep 2018 18:06:13 +0000 (14:06 -0400)]
ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Reworks the bluestore validation and reporting to account for reusable
VGs from fast devices, and adds validation calls to ensure the new way
to calculate this process will work.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27f47565a0d43ecee03dea4f6addc739e3e78655)

6 years agoceph-volume lvm.batch.filestore capture SizeAllocationErrors
Alfredo Deza [Fri, 7 Sep 2018 18:04:58 +0000 (14:04 -0400)]
ceph-volume lvm.batch.filestore capture SizeAllocationErrors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71fcd35c3d44795f92b17e9aedb7ca868b3750d3)

6 years agoceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore
Alfredo Deza [Thu, 6 Sep 2018 17:03:58 +0000 (13:03 -0400)]
ceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2ea49a4e64820fc3cd95c2a6ca8869309638e00)

6 years agoMerge pull request #23216 from b-ranto/wip-mgr-prom-bp-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:56:35 +0000 (14:56 -0700)]
Merge pull request #23216 from b-ranto/wip-mgr-prom-bp-luminous

luminous: Sync the prometheus module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #23461 from pdvian/wip-26838-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:54:37 +0000 (14:54 -0700)]
Merge pull request #23461 from pdvian/wip-26838-luminous

luminous: mgrc: enable disabling stats via mgr_stats_threshold

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #23566 from pdvian/wip-26908-luminous
Yuri Weinstein [Wed, 12 Sep 2018 21:53:52 +0000 (14:53 -0700)]
Merge pull request #23566 from pdvian/wip-26908-luminous

luminous: kv/KeyValueDB: return const char* from MergeOperator::name()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23579 from VictorDenisov/backport_26934
Yuri Weinstein [Wed, 12 Sep 2018 21:53:19 +0000 (14:53 -0700)]
Merge pull request #23579 from VictorDenisov/backport_26934

luminous: osd/OSDMap: fix apply upmap segfault

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #24039 from ceph/wip-s3a-fix-mirror
vasukulkarni [Wed, 12 Sep 2018 18:51:46 +0000 (11:51 -0700)]
Merge pull request #24039 from ceph/wip-s3a-fix-mirror

luminous: qa/tasks:  s3a fix mirror

6 years agoMerge pull request #23211 from neha-ojha/wip-24988-luminous
Yuri Weinstein [Wed, 12 Sep 2018 15:24:09 +0000 (08:24 -0700)]
Merge pull request #23211 from neha-ojha/wip-24988-luminous

luminous: osd: Limit pg log length during recovery/backfill so that we don't run out of memory

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agorgw: raise default rgw_curl_low_speed_time to 300 seconds 24046/head
Casey Bodley [Mon, 27 Aug 2018 18:45:35 +0000 (14:45 -0400)]
rgw: raise default rgw_curl_low_speed_time to 300 seconds

this timeout is mainly to avoid waiting forever when a sync connection
goes away without seeing a close. a 30-second timeout is much more
likely to catch other delays like loaded osds - and timeout/retry will
increase that load

Fixes: http://tracker.ceph.com/issues/27989
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2fe357657158f6454865b9c37314fa8cfdfc4771)

6 years agoMerge pull request #23761 from k0ste/luminous_backports3
Nathan Cutler [Tue, 11 Sep 2018 20:08:13 +0000 (22:08 +0200)]
Merge pull request #23761 from k0ste/luminous_backports3

luminous: docs: radosgw: ldap-auth: fixed option name 'rgw_ldap_searchfilter'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoqa/tasks: update mirror link for maven, the original mirror no longer exists 24039/head
Vasu Kulkarni [Thu, 6 Sep 2018 00:08:24 +0000 (17:08 -0700)]
qa/tasks: update mirror link for maven, the original mirror no longer exists

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 13e100259e3658fcfd2123474cb8ea8498ac3fb6)

6 years agoMerge pull request #23621 from joscollin/wip-26917-luminous
Nathan Cutler [Tue, 11 Sep 2018 16:26:09 +0000 (18:26 +0200)]
Merge pull request #23621 from joscollin/wip-26917-luminous

luminous: doc: Fix broken urls

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/rgw: override valgrind --max-threads for radosgw 24034/head
Casey Bodley [Wed, 1 Aug 2018 17:01:30 +0000 (13:01 -0400)]
qa/rgw: override valgrind --max-threads for radosgw

radosgw now uses 512 frontend threads by default, and valgrind won't
start with its default --max-threads=500

Fixes: http://tracker.ceph.com/issues/25214
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e617990efe4c39f592cff1541ea55fbcf9f2d6f2)

Conflicts:
qa/suites/rgw/tempest/tasks/rgw_tempest.yaml
qa/suites/rgw/verify/tasks/0-install.yaml

6 years agorgw: change default rgw_thread_pool_size to 512
Douglas Fuller [Fri, 15 Jun 2018 16:13:34 +0000 (12:13 -0400)]
rgw: change default rgw_thread_pool_size to 512

This value is commonly tuned up and demonstrates better performance on
large systems.

Fixes: http://tracker.ceph.com/issues/24544
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 28b0c1054fbc0d390d12fc32540c66b1e8c0bae4)

6 years agoMerge pull request #23967 from alfredodeza/luminous-rm35535
Andrew Schoen [Mon, 10 Sep 2018 14:29:22 +0000 (09:29 -0500)]
Merge pull request #23967 from alfredodeza/luminous-rm35535

luminous: ceph-volume: batch tests for mixed-type of devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorbd: fix error import when the input is a pipe 24003/head
songweibin [Fri, 31 Aug 2018 07:41:52 +0000 (15:41 +0800)]
rbd: fix error import when the input is a pipe

Fixes: http://tracker.ceph.com/issues/34536
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 3168d21629eb4d33f95dbd9399225e3e3817dc4f)

6 years agoosd/PrimaryLogPG: avoid dereferencing invalid complete_to 23211/head
xie xingguo [Mon, 3 Sep 2018 07:37:36 +0000 (15:37 +0800)]
osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

6 years agocls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder 23974/head
Vaibhav Bhembre [Thu, 31 May 2018 14:35:19 +0000 (10:35 -0400)]
cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Signed-off-by: Vaibhav Bhembre vaibhav@digitalocean.com
(cherry picked from commit a503cec93853235f2f94dbc53eb307618f5a437a)

6 years agoosd/PGPool::update: optimize with subset_of 23969/head
Zac Medico [Wed, 27 Sep 2017 01:33:15 +0000 (18:33 -0700)]
osd/PGPool::update: optimize with subset_of

Replace expensive inverval_set intersection_of and operator==
calls with a single subset_of call. I borrowed this idea from
Piotr Dałek's "osd/PGPool: don't use intermediate interval set"
patch. The following benchmark program demonstrates a 38%
performance increase:

#include <iostream>
#include <chrono>
#include "include/interval_set.h"

#define NANOSECONDS(d) \
    std::chrono::duration_cast<std::chrono::nanoseconds>(d).count()

typedef uint64_t snapid_t;
typedef std::chrono::steady_clock::duration duration;

duration PGPool_update_old(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;
    interval_set<snapid_t> intersection;
    intersection.intersection_of(newly_removed_snaps, cached_removed_snaps);

    assert(intersection == cached_removed_snaps);
    cached_removed_snaps.swap(newly_removed_snaps);
    newly_removed_snaps = cached_removed_snaps;
    newly_removed_snaps.subtract(intersection);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

duration PGPool_update_new(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;

    assert(cached_removed_snaps.subset_of(newly_removed_snaps));
    interval_set<snapid_t> removed_snaps = newly_removed_snaps;
    newly_removed_snaps.subtract(cached_removed_snaps);
    cached_removed_snaps.swap(removed_snaps);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

int main(int argc, char *argv[])
{
  assert(argc == 3);
  const int sample_count = std::stoi(argv[1]);
  const int interval_count = std::stoi(argv[2]);
  const int interval_distance = 4;
  const int interval_size = 2;
  const int max_offset = interval_count * interval_distance;
  interval_set<snapid_t> removed_snaps;

  for (int i = 0; i < max_offset; i += interval_distance)
    removed_snaps.insert(i, interval_size);

  duration old_delta(0), new_delta(0);

  for (int i = 0; i < sample_count; ++i) {
    old_delta += PGPool_update_old(removed_snaps);
    new_delta += PGPool_update_new(removed_snaps);
  }

  float ratio = float(NANOSECONDS(old_delta)) / NANOSECONDS(new_delta);

  std::cout << ratio << std::endl;
}

Suggested-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 18cbba4d2d7e26153afb01a3d904d5fb81cd4568)

6 years agoPGPool::update: optimize removed_snaps comparison when possible
Zac Medico [Thu, 31 Aug 2017 20:36:30 +0000 (13:36 -0700)]
PGPool::update: optimize removed_snaps comparison when possible

In self/unmanaged snaps mode, optimize removed_snaps comparison
for cases where removed_snaps has not changed. This exploits the
fact that remove_unmanaged_snap adds a dummy removed snapshot
to the end of removed_snaps, allowing for inexpensive detection
of changes. In cases where removed_snaps is very large, this
optimization improves performance dramatically.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit caf6803b13d9dbd4540da366b018d721fcfc371a)

6 years agoPGPool::update: avoid expensive union_of
Zac Medico [Thu, 24 Aug 2017 16:40:45 +0000 (09:40 -0700)]
PGPool::update: avoid expensive union_of

Achieve the same result using a swap (constant complexity),
and an assignment (linear complexity).

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit bf20b661cb6a6a33d9e93fb44ad1470f9f8245d2)

6 years agoPGPool::update: optimize with interval_set.swap
Zac Medico [Mon, 21 Aug 2017 20:30:53 +0000 (13:30 -0700)]
PGPool::update: optimize with interval_set.swap

Use constant complexity swap instead of linear complexity
assignment.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit ae64179a63351ac3966e70a074776188ddba5b84)

6 years agoceph-volume lvm.batch use 'ceph' as the cluster name with filestore 23967/head
Alfredo Deza [Thu, 6 Sep 2018 19:19:01 +0000 (15:19 -0400)]
ceph-volume lvm.batch use 'ceph' as the cluster name with filestore

Custom cluster names are currently broken on ceph-volume, should get
addressed with http://tracker.ceph.com/issues/27210 which is out of
scope for these changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1481799a254eceb05258313aafd532e062ea0bc)

6 years agoceph-volume tests/functional update filestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:30 +0000 (11:23 -0400)]
ceph-volume tests/functional update filestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a096a016ccd0acc86ce816aba8ce95f6e3b68c11)

6 years agoceph-volume tests/functional update bluestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:20 +0000 (11:23 -0400)]
ceph-volume tests/functional update bluestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89e52dd19792072919bef6ba2f2ab50d7b0ff4ed)

6 years agoceph-volume tests/functional update filestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:20 +0000 (11:18 -0400)]
ceph-volume tests/functional update filestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a5ec54207a02c4bb85dc8226890def18da45e2ff)

6 years agoceph-volume tests/functional update bluestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:05 +0000 (11:18 -0400)]
ceph-volume tests/functional update bluestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2549d5737283b39ecf5b0af98c73ffeb18033c36)

6 years agoceph-volume lvm.api use double -f flags when calling pvremove
Alfredo Deza [Thu, 6 Sep 2018 14:55:31 +0000 (10:55 -0400)]
ceph-volume lvm.api use double -f flags when calling pvremove

Fairly destructive, just like everything else when zapping a device.
This is required in the case of double UUIDs detected, something that
surfaced when testing with a loop device to create an nvme (the loop
device ends up with the same UUID as the nvme).

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1cbc407c095e0bc3e25a8ff1519b4909ec519cc)

6 years agoceph-volume tests/functional remove duplicate test playbooks
Alfredo Deza [Thu, 6 Sep 2018 14:26:05 +0000 (10:26 -0400)]
ceph-volume tests/functional remove duplicate test playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a008fe1385e4440f510eab97470a5d4e56a69402)

6 years agoceph-volume tests/functional setup playbook for nvme devices
Alfredo Deza [Thu, 6 Sep 2018 13:38:00 +0000 (09:38 -0400)]
ceph-volume tests/functional setup playbook for nvme devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71db509df0c4a14598010568f8bb078f6b2841ab)

6 years agoceph-volume tests/functional single (test) playbook to zap and redeploy batch operations
Alfredo Deza [Thu, 6 Sep 2018 13:34:11 +0000 (09:34 -0400)]
ceph-volume tests/functional single (test) playbook to zap and redeploy batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 23330c0107a9f9ee436338c112b1852fd36e34a3)

6 years agoceph-volume tests/functional create a noop playbook to allow symlinking always to...
Alfredo Deza [Thu, 6 Sep 2018 13:33:21 +0000 (09:33 -0400)]
ceph-volume tests/functional create a noop playbook to allow symlinking always to setup.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860e66083fe769714a10076fa101b94507e47169)

6 years agoceph-volume tests/functional add mixed-type scenarios to tox.ini
Alfredo Deza [Thu, 6 Sep 2018 13:32:00 +0000 (09:32 -0400)]
ceph-volume tests/functional add mixed-type scenarios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d5b90bb1fbf30c6e33dbf7e10a3d0ba053297ae)

6 years agoceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:54 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 799ce4508d9ed166ef6447daa21e0564262fbafe)

6 years agoceph-volume tests/functional add CentOS7 filestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:38 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 filestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89bbcff659eed8f2cfa26fb93a0c12c7ff83e83b)

6 years agomsg: lower verbosity on normal event 23943/head
Patrick Donnelly [Sun, 2 Sep 2018 19:51:46 +0000 (12:51 -0700)]
msg: lower verbosity on normal event

Fixes: http://tracker.ceph.com/issues/35251
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 351831834d4f9c66395539d7015bcdb017c8dba6)

Conflicts:
src/msg/async/AsyncConnection.cc
src/msg/simple/Pipe.cc

6 years agoMerge pull request #23913 from yuriw/wip-35541-luminous
Nathan Cutler [Wed, 5 Sep 2018 18:28:17 +0000 (20:28 +0200)]
Merge pull request #23913 from yuriw/wip-35541-luminous

qa/tests:  added v12.2.8 to the mix

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/tests: added v12.2.8 to the mix 23913/head
Yuri Weinstein [Tue, 4 Sep 2018 16:05:42 +0000 (09:05 -0700)]
qa/tests:  added v12.2.8 to the mix

Fixes: http://tracker.ceph.com/issues/35541
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agotest: Use pids instead of jobspecs which were wrong 23901/head
David Zafman [Thu, 16 Aug 2018 21:30:09 +0000 (14:30 -0700)]
test: Use pids instead of jobspecs which were wrong

Fixes: http://tracker.ceph.com/issues/27056
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bc3317031057b5737710a095ce593135b62f9302)

6 years agolibrbd: potential race on image create request complete 23892/head
Mykola Golub [Sat, 18 Aug 2018 16:24:55 +0000 (19:24 +0300)]
librbd: potential race on image create request complete

Fixes: http://tracker.ceph.com/issues/24910
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d9e5cfaf4a8cd090362892805e837f777b220474)

6 years agotest: Update tests for error message changes 23871/head
David Zafman [Thu, 16 Aug 2018 22:31:53 +0000 (15:31 -0700)]
test: Update tests for error message changes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 687f63e59970433bfb53c17d4128a3d042a0c9c6)

Conflicts:
qa/standalone/scrub/osd-scrub-snaps.sh

- luminous has two additional errors

6 years agoosd: Use colon to separate error message from other info
David Zafman [Thu, 16 Aug 2018 22:24:11 +0000 (15:24 -0700)]
osd: Use colon to separate error message from other info

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bdd32cc84787892b3005f2644b5ad9f8ac7e772d)

6 years agoqa: Add new message to whitelist for scrub/repair tests
David Zafman [Mon, 6 Aug 2018 17:37:04 +0000 (10:37 -0700)]
qa: Add new message to whitelist for scrub/repair tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b40784290f3b600941e0d97372fddee5c85ec20c)

6 years agotest: Verify cluster logging of scrub error messages
David Zafman [Fri, 3 Aug 2018 20:47:30 +0000 (13:47 -0700)]
test: Verify cluster logging of scrub error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 58c4d322032d04b0d55f627ce42531ca3edab3e9)

6 years agoosd: Log shard errors in be_select_auth_object() as other errors
David Zafman [Wed, 1 Aug 2018 22:13:40 +0000 (15:13 -0700)]
osd: Log shard errors in be_select_auth_object() as other errors

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cadf727258ea8b3a52ae9122b42a0b4f6914e177)

Conflicts:
src/osd/PGBackend.cc

- luminous still has oi_prio

6 years agoosd: Move object error check out of be_select_auth_object()
David Zafman [Wed, 1 Aug 2018 00:53:17 +0000 (17:53 -0700)]
osd: Move object error check out of be_select_auth_object()

Fixes: http://tracker.ceph.com/issues/25108
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c789fd1899f697a994a28e45a6f9262c6721d77d)

Conflicts:
src/osd/PGBackend.cc

- luminous still has auth_prio and different way of decoding bliter

6 years agopybind/rados/rados: do not pass prval from stack 23864/head
Sage Weil [Mon, 30 Jul 2018 19:18:07 +0000 (14:18 -0500)]
pybind/rados/rados: do not pass prval from stack

The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8e36f18cdeaa2088a6ce4aaad61b76283e777270)

6 years agoosd,mon: increase mon_max_pg_per_osd to 250 23862/head
Neha Ojha [Thu, 26 Jul 2018 00:25:35 +0000 (17:25 -0700)]
osd,mon: increase mon_max_pg_per_osd to 250

https://bugzilla.redhat.com/show_bug.cgi?id=1603615 indicates
a case when pg calc conflicts with mon_max_pg_per_osd, and does not
allow pool creation when this limit is 200. Hence, increase this limit
to avoid this.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f7911de61bca7c7f1ef007eb229ee845923c9751)