]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Alfredo Deza [Fri, 3 Nov 2017 18:23:47 +0000 (14:23 -0400)]
ceph-volume simple add __init__ for the module
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
285e5bdd28a42d98418ca90ad0cdec6bd87849db )
Alfredo Deza [Fri, 3 Nov 2017 18:23:29 +0000 (14:23 -0400)]
ceph-volume simple create the initial, main module
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
f050502425a1993ae8edfdb5785b24129cffeeba )
Abhishek L [Thu, 9 Nov 2017 17:08:27 +0000 (18:08 +0100)]
Merge pull request #18801 from linuxbox2/luminous-rgw-readdir2
rgw_file: implement variant offset readdir processing -- pullup 2 of 2
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Kefu Chai [Thu, 9 Nov 2017 14:36:27 +0000 (22:36 +0800)]
Merge pull request #18734 from shinobu-x/wip-21648-luminous
luminous: mgr[zabbix] float division by zero
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Kefu Chai [Thu, 9 Nov 2017 14:35:44 +0000 (22:35 +0800)]
Merge pull request #18851 from tchaikov/wip-18759-pr-luminous
mon/LogMonitor: "log last" should return up to n entries
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Thu, 9 Nov 2017 14:27:47 +0000 (22:27 +0800)]
Merge pull request #18741 from shinobu-x/wip-22019-luminous
luminous: tests: "ceph osd create" is not idempotent
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 6 Nov 2017 08:01:23 +0000 (16:01 +0800)]
mon/LogMonitor: "log last" should return up to n entries
limit the # of returned entries to "num", and backoff the start iterator
by one if it reaches the rend().
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
de14103f5201da69b360d9b01e249896d59376b3 )
Matt Benjamin [Mon, 16 Oct 2017 01:48:19 +0000 (21:48 -0400)]
rgw_file: implement variant offset readdir processing
Introduce new rgw_readdir2(...), which in which continues
from an arbitrary dirent name, which presumably has been
seen in a prior partial enumeration.
Add single-file unit test for READDIR cases, librgw_file_marker.cc.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
a2c34c597371809bd8fd40ee1fc6b8e6a5145609 )
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Abhishek L [Thu, 9 Nov 2017 09:10:53 +0000 (10:10 +0100)]
Merge pull request #18800 from linuxbox2/luminous-rgw_file-fsid-mount
Luminous rgw file fsid mount -- pullup 1 of 2
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Abhishek L [Thu, 9 Nov 2017 09:04:08 +0000 (10:04 +0100)]
Merge pull request #18767 from smithfarm/wip-22020-luminous
luminous: multisite: race between sync of bucket and bucket instance metadata
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Abhishek L [Thu, 9 Nov 2017 08:56:16 +0000 (09:56 +0100)]
Merge pull request #18794 from theanalyst/wip-18709
luminous: rgw: Stale bucket index entry remains after object deletion
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 10:19:27 +0000 (18:19 +0800)]
Merge pull request #18726 from shinobu-x/wip-luminous-22035
luminous: Spurious ceph-mgr failovers during mon elections
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 10:18:39 +0000 (18:18 +0800)]
Merge pull request #18738 from shinobu-x/wip-21547-luminous
luminous: ceph-mgr gets process called "exe" after respawn
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 10:17:54 +0000 (18:17 +0800)]
Merge pull request #18727 from shinobu-x/wip-luminous-22034
luminous: key mismatch for mgr after upgrade from jewel to luminous(dev)
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 06:44:41 +0000 (14:44 +0800)]
Merge pull request #18723 from shinobu-x/wip-luminous-22023
luminous: osd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL
Kefu Chai [Wed, 8 Nov 2017 06:38:10 +0000 (14:38 +0800)]
Merge pull request #18737 from shinobu-x/wip-21549-luminous
luminous: the dashboard uses absolute links for filesystems and clients
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 06:37:19 +0000 (14:37 +0800)]
Merge pull request #18736 from shinobu-x/wip-21638-luminous
luminous: dashboard OSD list has servers and osds in arbitrary order
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 06:36:40 +0000 (14:36 +0800)]
Merge pull request #18728 from shinobu-x/wip-luminous-22032
luminous: dashboard barfs on nulls where it expects numbers
Reviewed-by: John Spray <john.spray@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 06:34:35 +0000 (14:34 +0800)]
Merge pull request #18699 from shinobu-x/wip-luminous-21538
luminous: upmap does not respect osd reweights
Reviewed-by: Sage Weil <sage@redhat.com>
Kefu Chai [Wed, 8 Nov 2017 06:30:42 +0000 (14:30 +0800)]
Merge pull request #18719 from tchaikov/wip-pr-18373-luminous
luminous: osd,mon: add max-pg-per-osd limit
Reviewed-by: Sage Weil <sage@redhat.com>
Kefu Chai [Wed, 18 Oct 2017 08:07:50 +0000 (16:07 +0800)]
doc: fix the desc of "osd max pg per osd hard ratio"
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
d940f6220346ad21dfc4f351c57389e8430c0257 )
Kefu Chai [Tue, 17 Oct 2017 04:17:09 +0000 (12:17 +0800)]
mon: change "mon_pg_warn_min_per_osd" to uint64_t
* this silences a warning from -Wsign-compare.
* also switch all its user to the new-style option
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
c3cd0c13a1585388df0f17ab53c170e39c711615 )
Sage Weil [Wed, 8 Nov 2017 04:27:15 +0000 (22:27 -0600)]
Merge pull request #18702 from liewegas/wip-pr-18670-luminous
luminous: qa/tasks/thrashosds: set min_in default to 4
Abhishek L [Tue, 7 Nov 2017 21:28:58 +0000 (22:28 +0100)]
Merge pull request #18596 from smithfarm/wip-21955-luminous
luminous: tests: add EC data pool to testing
Reviewed-By: Patrick Donelly <pdonelly@redhat.com>
Gui Hecheng [Sat, 22 Jul 2017 05:12:06 +0000 (13:12 +0800)]
rgw_file: introduce rgw_mount2 with a bucket name parameter
This explicitly allow mount a specified bucket in librgw.
Originally, mounting a bucket is implemented like a sub-directory
mount in nfs-ganesha FSAL_RGW with lookup_path.
With this change, we allow root_fh to points exactly to the root
of a mounted fs instance, which is a bucket or "/" rather than
always let root_fh points to "/".
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit
860716e17e840df11a7e1d8b6ca0c1ee694b038b )
Gui Hecheng [Fri, 26 May 2017 09:45:29 +0000 (17:45 +0800)]
rgw_file: new fsid for fs instance
We use an in-memory fs_inst_counter for fsid currently, but it
inevitably cause nfs automount problem across a nfs-ganesha crash.
So here we relate the fsid to the s3 uid with a hash.
With this, fsid remains consistent across all fs instances and
across crashes.
We should be able to upgrade from old fsid to new fsid with
update_fh().
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit
178b547e83a2c0eeb024bbe4ca2ff8d8e7d50572 )
Abhishek L [Tue, 7 Nov 2017 21:07:45 +0000 (22:07 +0100)]
Merge pull request #18628 from batrick/i21953
luminous: mds: sanitize mdsmap of removed pools
Reviewed-By: Sage Weil <sage@redhat.com>
Abhishek L [Tue, 7 Nov 2017 17:33:21 +0000 (18:33 +0100)]
Merge pull request #18626 from dillaman/wip-21968
luminous: cls/journal: fixed possible infinite loop in expire_tags
Reviewed-By: Mykola Golub <trociny@gmail.com>
Abhishek L [Tue, 7 Nov 2017 17:32:20 +0000 (18:32 +0100)]
Merge pull request #18688 from liewegas/wip-balancer-luminous
luminous: mgr/balancer: backport crush-compat work from master
Reviewed-By: John Spray <jspray@redhat.com>
J. Eric Ivancich [Fri, 3 Nov 2017 13:15:13 +0000 (09:15 -0400)]
rgw: fix BZ
1500904 , Stale bucket index entry remains after object deletion
We have a race condition:
1. RGW client #1: requests an object be deleted.
2. RGW client #1: sends a prepare op to bucket index OSD #1.
3. OSD #1: prepares the op, adding pending ops to the bucket dir entry
4. RGW client #2: sends a list bucket to OSD #1
5. RGW client #2: sees that there are pending operations on bucket
dir entry, and calls check_disk_state
6. RGW client #2: check_disk_state sees that the object still exists, so it
sends CEPH_RGW_UPDATE to bucket index OSD (#1)
7. RGW client #1: sends a delete object to object OSD (#2)
8. OSD #2: deletes the object
9. RGW client #2: sends a complete op to bucket index OSD (#1)
10. OSD #1: completes the op
11. OSD #1: receives the CEPH_RGW_UPDATE and updates the bucket index
entry, thereby **RECREATING** it
Solution implemented:
At step #5 the object's dir entry exists. If we get to beginning of
step #11 and the object's dir entry no longer exists, we know that the
dir entry was just actively being modified, and ignore the
CEPH_RGW_UPDATE operation, thereby NOT recreating it.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit
b33f529e79b74314a2030231e1308ee225717743 )
Abhishek L [Tue, 7 Nov 2017 17:21:41 +0000 (18:21 +0100)]
Merge pull request #18569 from kmroz/wip-21939-luminous
luminous: list bucket which enable versioning get wrong result when user marker
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Abhishek L [Tue, 7 Nov 2017 17:21:03 +0000 (18:21 +0100)]
Merge pull request #18591 from cbodley/wip-21938
luminous: rgw: fix error handling in ListBucketIndexesCR
(approved in irc & standups)
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Abhishek L [Tue, 7 Nov 2017 17:17:57 +0000 (18:17 +0100)]
Merge pull request #18539 from linuxbox2/luminous-baixuyeyu-refcnt
luminous: baixuyeyu refcnt
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Abhishek L [Tue, 7 Nov 2017 17:15:38 +0000 (18:15 +0100)]
Merge pull request #18516 from ceph/wip-bp-pr18193-luminous
qa/rgw: ignore errors from 'pool application enable'
Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Abhishek L [Tue, 7 Nov 2017 17:14:21 +0000 (18:14 +0100)]
Merge pull request #18599 from linuxbox2/lum-rgwfile-21940
rgw_file: set s->obj_size from bytes_written
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Abhishek L [Tue, 7 Nov 2017 12:07:09 +0000 (13:07 +0100)]
Merge pull request #18566 from kmroz/wip-21914-luminous
luminous: [rbd-mirror] peer cluster connections should filter out command line optionals
Reviewed-By: Jason Dillaman <jdillaman@redhat.com>
Kefu Chai [Tue, 17 Oct 2017 04:10:55 +0000 (12:10 +0800)]
osd,mon: change "mon_max_pg_per_osd" to uint64_t
as it should never be a negative number. if this option is disabled,
it's 0.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
0d68197376b6cf48c6679d77ec9f28f279f3da68 )
Kefu Chai [Tue, 26 Sep 2017 07:54:14 +0000 (15:54 +0800)]
osd: add max-pg-per-osd limit
osd will refused to create new pgs, until its pg number is lower
than the max-pg-per-osd upper bound setting.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
4c7df944c7f28232873ba681eedce72cdb062ea5 )
Kefu Chai [Sun, 15 Oct 2017 12:46:28 +0000 (20:46 +0800)]
qa/tasks/ceph: add "create_rbd_pool" option
True by default, but we can opt not to create the "rbd" pool.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
ac1e13ce63a6f32ae9ab0db8196c8cc8100d94ed )
Kefu Chai [Thu, 12 Oct 2017 12:07:46 +0000 (20:07 +0800)]
osd: print osd as "osd.$osdid" in log message
easier to read this way
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
c6b71ebf6b7b7c54560f331d5f8e4e24f5a60633 )
Kefu Chai [Tue, 26 Sep 2017 07:50:08 +0000 (15:50 +0800)]
mon/OSDMonitor: bypass checks if pg_temp.forced
originally, monitor ignores the pg_temp messages sent from replica osds,
but to re-trigger peering if replica osd's pg number drops down below
the setting, we need to bypass the checks.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
da6f7a810055a0dc9e364aca8e6efbfbc32881b3 )
Kefu Chai [Tue, 26 Sep 2017 07:49:11 +0000 (15:49 +0800)]
messages/MOSDPGTemp: add `forced` field
the monitor is supposed to bypass some checks if `forced` is true.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
6335bf791003e9e33b2f738ab3da06f3c626537a )
Kefu Chai [Tue, 7 Nov 2017 09:48:34 +0000 (17:48 +0800)]
Merge pull request #18775 from tchaikov/wip-upgrade-luminous-x
qa: add 1-ceph-install to upgrade:luminous-x
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Yuri Weinstein [Tue, 29 Aug 2017 16:50:51 +0000 (09:50 -0700)]
qa: add 1-ceph-install to upgrade:luminous-x
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit
aff342627cf77e2faae6904e626d9fb2c897f155 )
Sage Weil [Tue, 7 Nov 2017 03:17:22 +0000 (21:17 -0600)]
qa/suites/rados/thrash/d-*/*balancer*: debug osd in mgr
Signed-off-by: Sage Weil <sage@redhat.com>
Casey Bodley [Tue, 31 Oct 2017 20:56:01 +0000 (16:56 -0400)]
rgw: remove placement_rule from cls_user_bucket_entry
placement_rule is no longer needed in cls_user_bucket_entry, because the
only time that it's needed, we can read it from the bucket instance in
RGWRados::update_containers_stats()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
8e62e3526643da67f5af7daa687120feed469785 )
Casey Bodley [Tue, 31 Oct 2017 20:29:31 +0000 (16:29 -0400)]
rgw: remove placement_rule from rgw_link_bucket()
with the fallback in RGWRados::update_containers_stats(), we no longer
need to pass placement_rule into the cls_user_bucket_entry
this removes the dependency between bucket entrypoint metadata and its
bucket instance metadata during multisite sync
Fixes: http://tracker.ceph.com/issues/21990
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
dd5d278a4f9b080234cbb77e448208803ec0cf93 )
Casey Bodley [Tue, 31 Oct 2017 20:26:54 +0000 (16:26 -0400)]
rgw: take placement_rule from bucket info in update_containers_stats
in case the cls_user_bucket_entry doesn't contain a placement_rule, take
it from the bucket instance info
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
531195d80d5bac774d58b210307eb19b57d26345 )
Sage Weil [Sat, 4 Nov 2017 14:32:51 +0000 (09:32 -0500)]
qa/suites/rados/thrash: combine balancer with require-luminous
We can only do the balancer stuff if we've done the mkfs-time
require luminous because otherwise the mgr commands aren't proxied via
the mon.
Only do balancer for thrash tests; not for basic, monthrash, verify.
Signed-off-by: Sage Weil <sage@redhat.com>
Kefu Chai [Wed, 1 Nov 2017 02:45:09 +0000 (10:45 +0800)]
qa: stop testing deprecated "ceph osd create"
"ceph osd create" is not idempotent, and is considered deprecated.
Fixes: http://tracker.ceph.com/issues/21993
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
371d3f40911917704f4992c48236b4c60905bf4e )
John Spray [Fri, 15 Sep 2017 16:23:39 +0000 (12:23 -0400)]
mgr: set explicit thread name
This gets used as our process name in some situations
when respawning. This is the same as what commit
4f177bb6b
did for the MDS.
Fixes: http://tracker.ceph.com/issues/21404
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
44dce0a5664639b2ae0949cc434f1be1176d872d )
Nick Erdmann [Tue, 22 Aug 2017 16:55:11 +0000 (16:55 +0000)]
pybind/mgr/dashboard: updated favicon (old one was 404)
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit
7a54520f5c3d4db27fe7e2a12760dcd838ba89aa )
Nick Erdmann [Tue, 22 Aug 2017 08:08:45 +0000 (08:08 +0000)]
pybind/mgr/dashboard: clean up
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit
f512ac3d7712850f274cd01bb589aa9fbc8c8bef )
Nick Erdmann [Mon, 21 Aug 2017 17:21:10 +0000 (17:21 +0000)]
pybind/mgr/dashboard: add url_prefix
This adds a configuration variable url_prefix to the dashboard that
that is prepended to all URLs so you can access the dashboard at
http://$IP:$PORT/$PREFIX/. This is necessary if you wish to use a
reverse http proxy that forwards to the dashboard under a sub-path.
Fixes: http://tracker.ceph.com/issues/20568
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit
6d285fff0df598d66f46d38bf0a6a2cc9c0dc62f )
John Spray [Wed, 27 Sep 2017 14:13:48 +0000 (15:13 +0100)]
mgr/dashboard: sort servers and OSDs in OSD list
Fixes: http://tracker.ceph.com/issues/21572
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
f409099fa25b73fe580ee4662aff51636f118ed2 )
John Spray [Sat, 23 Sep 2017 14:22:34 +0000 (15:22 +0100)]
mgr/zabbix: log exceptions with backtrace
Logging just the exception object gave a sometimes-enigmatic
single line. Let's use the logger exception method so that
we get a backtrace.
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
316dcc8f0b42b85503632d5472c45c9828882f2c )
John Spray [Sat, 23 Sep 2017 14:18:18 +0000 (15:18 +0100)]
mgr/zabbix: fix div by zero
Fixes: http://tracker.ceph.com/issues/21518
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
9c02738b4ea0303f5b3cbfc4748d6791007be834 )
Sage Weil [Sat, 4 Nov 2017 15:52:27 +0000 (10:52 -0500)]
Merge pull request #18673 from dzafman/wip-21833
osd: build_past_intervals_parallel: Ignore new partially created PGs
John Spray [Wed, 27 Sep 2017 13:33:58 +0000 (14:33 +0100)]
mgr/dashboard: handle null in format_number
Fixes: http://tracker.ceph.com/issues/21570
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
f43859d4dca696ba4cac224e0c7e87e4b2aad40f )
John Spray [Thu, 19 Oct 2017 13:28:18 +0000 (09:28 -0400)]
mon: don't blow away bootstrap-mgr on upgrades
Fixes: http://tracker.ceph.com/issues/20950
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
57229ea2a4369518c7a16b7a09b045b7896f5a70 )
John Spray [Thu, 12 Oct 2017 10:57:50 +0000 (11:57 +0100)]
mon: handle monitor lag when killing mgrs
Fixes: http://tracker.ceph.com/issues/20629
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
3c3776b30a2da3f5d2ece15c57713c0ce191c778 )
Yao Zongyou [Tue, 31 Oct 2017 05:23:49 +0000 (13:23 +0800)]
osd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL
These two counters should be reported to mgr because the dashboard
plugin needing them to display osd detailed information.
Fixes: http://tracker.ceph.com/issues/21981
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit
40bb3c5f05b6f2a917a7e400f88677f41a9629a3 )
Sage Weil [Thu, 2 Nov 2017 21:18:03 +0000 (16:18 -0500)]
pybind/mgr/mgr_module: fix calc_pg_upmaps
This was fixed in the object cleanup.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
ed442164d52c803916ca43a5ed76047089d3da66 )
Sage Weil [Thu, 2 Nov 2017 21:11:26 +0000 (16:11 -0500)]
mgr/balancer: enable module by default
It will still be "off".
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
26710f0a9b176289237a52e5fa5894342ad163dc )
Sage Weil [Wed, 1 Nov 2017 13:31:31 +0000 (08:31 -0500)]
qa/tasks/thrashosds: set min_in default to 4
We have EC tests with k=2,m=2, so we need a min of 4.
Fixes: http://tracker.ceph.com/issues/21997
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
d21809b14ea58dc1f44df844e407ebab5a315062 )
Abhishek L [Fri, 3 Nov 2017 09:26:44 +0000 (10:26 +0100)]
Merge pull request #18446 from theanalyst/wip-21854-luminous
luminous: rgw_file: explicit NFSv3 open() emulation
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>
Theofilos Mouratidis [Mon, 25 Sep 2017 14:17:47 +0000 (16:17 +0200)]
osd: upmap should respect osd reweights
Modify OSDMap::calc_pg_upmaps to take the osd reweight into account when
computing the size of each OSD.
Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
Fixes: http://tracker.ceph.com/issues/21538
(cherry picked from commit
b8561cefa705d31954d47723125376a3c087f4f4 )
Theofilos Mouratidis [Mon, 25 Sep 2017 14:15:45 +0000 (16:15 +0200)]
test/cli/osdmaptool: test upmap with an out osd
Test upmap with one osd marked out.
Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit
ff9e024a59d87eda6cbf3be9464ee7ae6f1140d7 )
Theofilos Mouratidis [Mon, 25 Sep 2017 14:14:09 +0000 (16:14 +0200)]
osdmaptool: add --mark-out option
Add a --mark-down <id> option which sets the reweight for a given
osd id to 0.0.
Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit
bd75a78187143f750f90fb937984eaa6333159d3 )
Abhishek L [Thu, 2 Nov 2017 20:23:50 +0000 (21:23 +0100)]
Merge pull request #18439 from theanalyst/wip-21655-luminous
luminous: expose --sync-stats via admin api
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Sage Weil [Fri, 27 Oct 2017 17:53:52 +0000 (12:53 -0500)]
mgr/balancer: simplify pool_info tracking
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
a59a1df85382d559362c51bb45284b61caaca1d4 )
Sage Weil [Thu, 26 Oct 2017 03:47:02 +0000 (22:47 -0500)]
mgr/balancer: less verbose on 'eval' by default; add 'eval-verbose'
The verbose output is helpful for debugging and understanding what is
being measured, but most of the time all that you care about is the score.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2c300bc8d7189b464f59c93deb5eb56915df62b8 )
Sage Weil [Sat, 21 Oct 2017 15:46:27 +0000 (10:46 -0500)]
mgr/balancer: fix pg vs object terminology
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
6513e8b0fc2ebcbbbda937ec6d533ea084ef278f )
Sage Weil [Tue, 24 Oct 2017 22:12:54 +0000 (17:12 -0500)]
mgr/balancer: restrict to time of day
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
3f0b50b50aeb34ed39767289b8475ec5b877cc27 )
Sage Weil [Thu, 5 Oct 2017 22:31:06 +0000 (17:31 -0500)]
mgr/module: adjust osd_weight min step to .005
That should be ~1 PG or less on average; no real sense it taking a
step smaller than that!
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
297e2d65abcebe45c6be69a768cf3ea98e1b354d )
Sage Weil [Thu, 5 Oct 2017 22:26:16 +0000 (17:26 -0500)]
mgr/balancer: if score regresses, take a few more steps
To avoid getting stuck in a local minima, take a few additional
steps if our score drops and see if it gets better.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
70b503495a0ceb7d1aa4f5fce4403a8eff9a03b5 )
Sage Weil [Thu, 5 Oct 2017 22:25:19 +0000 (17:25 -0500)]
mgr/balancer: allow 5% misplaced
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
1ad3101818d1855c8eea8386d7682fab2fe9c8b8 )
Sage Weil [Thu, 5 Oct 2017 22:25:06 +0000 (17:25 -0500)]
mgr/balancer: more aggressive steps
We are smart enough to back off if we misplace too much.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
0dbcf737eaed02b5aa94bc6f27231b4b87f09ce5 )
Sage Weil [Sun, 1 Oct 2017 21:00:06 +0000 (16:00 -0500)]
qa/suites/rados/thrash/d-balancer: enable balancer in various modes
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2c9c18d1ec3d33d385adf2a30dc1d3b8e987f9a5 )
Sage Weil [Wed, 27 Sep 2017 21:05:05 +0000 (17:05 -0400)]
mgr/balancer: crush-compat: phase out osd_weights
Phase out the osd_weight values as we optimize the crush compat
weight set. Allow a small regression in overall score if it means
we are taking a step to remove the osd_weight value.
Fix the 'osd reweightn' invocation.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
861c37fd99a76d5f1665a610224778fcbb41e9e1 )
Sage Weil [Wed, 27 Sep 2017 19:12:54 +0000 (15:12 -0400)]
mgr/balancer: crush_compat: cope with 'out' osds
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
c2bba5820a0bf2225dfc9ba39cc67c78083aba11 )
Sage Weil [Wed, 27 Sep 2017 19:12:36 +0000 (15:12 -0400)]
mgr/balancer: stop if we get a perfect score
..and fix the 'worse' threshold (we're just working around floating
point imprecision; 1.01 was too coarse).
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
f1f3014ffcc67200c20e6fb2b2e6e8f1141a0a1d )
Sage Weil [Wed, 27 Sep 2017 15:40:36 +0000 (11:40 -0400)]
mgr/balancer: more dead code
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
37bd64405b8a578329b9a903c9a53322ea29caaf )
Sage Weil [Wed, 27 Sep 2017 15:27:42 +0000 (11:27 -0400)]
mgr/balancer: crush-compat: throttle changes based on max_misplaced
Take smaller steps if we overshoot max_misplaced.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
df2e008d3388b657d62ea4547e6b1b7a0251aa04 )
Sage Weil [Wed, 27 Sep 2017 15:19:30 +0000 (11:19 -0400)]
mgr/balancer: remove dead code
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
020be37fd06a75ec86e05af966be318c2199da9f )
Sage Weil [Wed, 27 Sep 2017 15:19:23 +0000 (11:19 -0400)]
mgr/balancer: include pg up mapping in MappingState
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
f6f2d253c379a6235e810808e7ce3b83cc4e212c )
Sage Weil [Tue, 26 Sep 2017 22:36:06 +0000 (18:36 -0400)]
mgr/balancer: normalize weight-set weights to sum to target weight
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
99fcd161bd222a4f0d2ddde0f1a4caecd1a78882 )
Sage Weil [Tue, 26 Sep 2017 22:35:42 +0000 (18:35 -0400)]
mgr/balancer: note root id in Eval
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
26a7ec0c8a4c215791369d8650cd17d41317aefe )
Sage Weil [Tue, 26 Sep 2017 22:00:08 +0000 (18:00 -0400)]
mgr/balancer: make crush-compat mode work!
- it does multiple iterations, like the upmap optimizer.
- it decreases the step size if it isn't improving, in the hope
that it is overshooting the minimum
- debug output is cleaned up a bit (the info level should be
genuinely useful)
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
d9a31595ba15de5fda104a0154778e3200fc46a0 )
Sage Weil [Thu, 2 Nov 2017 19:54:50 +0000 (14:54 -0500)]
Merge pull request #18590 from liewegas/wip-prime-fixes-luminous
luminous: ceph-bluestore-tool: prime-osd-dir: update symlinks instead of bailing
Sage Weil [Thu, 2 Nov 2017 19:54:18 +0000 (14:54 -0500)]
Merge pull request #18620 from tchaikov/wip-21527-luminous
luminous: mon/mgr: sync "mgr_command_descs","osd_metadata" and "mgr_metadata" prefixes to new mons
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Sage Weil [Thu, 2 Nov 2017 19:53:16 +0000 (14:53 -0500)]
Merge pull request #18621 from tchaikov/wip-21534-luminous
luminous: mon: update get_store_prefixes implementations
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Sage Weil [Thu, 2 Nov 2017 19:52:32 +0000 (14:52 -0500)]
Merge pull request #18623 from liewegas/wip-pr-18585-luminous
luminous: ceph-bluestore-tool: prime-osd-dir: update symlinks instead of bailing
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Thu, 2 Nov 2017 19:51:58 +0000 (14:51 -0500)]
Merge pull request #18625 from tchaikov/wip-ceph-disk-unlock-dmcrypted-partitions-luminous
luminous: ceph-disk: unlocks dmcrypted partitions when activating them
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Sage Weil [Thu, 2 Nov 2017 19:51:32 +0000 (14:51 -0500)]
Merge pull request #18650 from b-ranto/wip-allow-getattr-luminous
luminous: selinux: Allow getattr on lnk sysfs files
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Sage Weil [Thu, 2 Nov 2017 19:51:11 +0000 (14:51 -0500)]
Merge pull request #18675 from jcsp/wip-luminous-mgr
luminous: bulk backport of ceph-mgr improvements
Alfredo Deza [Thu, 2 Nov 2017 17:39:05 +0000 (13:39 -0400)]
Merge pull request #18687 from ceph/backport-18656
luminous: "ceph-volume: adds functional testing for bluestore"
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Kefu Chai [Thu, 2 Nov 2017 17:02:26 +0000 (01:02 +0800)]
Merge pull request #18293 from smithfarm/wip-21795-luminous
luminous: Ubuntu amd64 client can not discover the ubuntu arm64 ceph cluster
Reviewed-by: Kefu Chai <kchai@redhat.com>
Andrew Schoen [Tue, 31 Oct 2017 20:25:23 +0000 (15:25 -0500)]
ceph-volume tests: adds objecstore as a testing factor
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
f8b114be61d033c853a2e8295b966e30c6a5f721 )
Andrew Schoen [Tue, 31 Oct 2017 20:10:28 +0000 (15:10 -0500)]
ceph-volume: enable the centos7-bluestore and xenial-bluestore tests
This also updates the ansible and testinfra version to match the current
ceph-ansible master branch.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
b49a19a9773599de295197979b498d06ca500cf8 )