]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph-volume lvm.activate auto detect objectstore type
Alfredo Deza [Wed, 18 Oct 2017 19:06:01 +0000 (15:06 -0400)]
ceph-volume lvm.activate auto detect objectstore type

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 472aa0b1997a0b1301b699b7ce30dce728aaea0c)

7 years agoceph-volume lvm.trigger enable objectstore auto detection for activate
Alfredo Deza [Wed, 18 Oct 2017 19:00:19 +0000 (15:00 -0400)]
ceph-volume lvm.trigger enable objectstore auto detection for activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f579ef07026564f1a94b2923a2e4640ade2da503)

7 years agoceph-volume lvm.activate add bluestore support for activation
Alfredo Deza [Wed, 18 Oct 2017 16:00:34 +0000 (12:00 -0400)]
ceph-volume lvm.activate add bluestore support for activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5696fed09b65044142b34f89492a6d91d3243e14)

7 years agoceph-volume util.system create mappings of mounts
Alfredo Deza [Wed, 18 Oct 2017 15:40:38 +0000 (11:40 -0400)]
ceph-volume util.system create mappings of mounts

This makes it easier for consumers to detect a path, or a device that
might be mounted or might have more than one mount, including tmpfs and
devtmpfs support, which was just not possible before.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f61007cc650f6964ef2ff3ea98a1debf02713198)

7 years agoceph-volume util.prepare separate filestore vs. bluestore prepare utils
Alfredo Deza [Mon, 16 Oct 2017 10:51:05 +0000 (06:51 -0400)]
ceph-volume util.prepare separate filestore vs. bluestore prepare utils

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 85cdc783008ff0e0566c34aaab7cd5f12db1cbc7)

7 years agoceph-volume process allow obfuscating a key or index when logging commands
Alfredo Deza [Mon, 16 Oct 2017 10:50:27 +0000 (06:50 -0400)]
ceph-volume process allow obfuscating a key or index when logging commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bbcca70d962855d991f0aa8e86a19f1855345668)

7 years agoceph-volume util.prepare add other ln helpers for bluestore
Alfredo Deza [Fri, 13 Oct 2017 13:22:53 +0000 (09:22 -0400)]
ceph-volume util.prepare add other ln helpers for bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 555b664882db2cd247087f907bdf4d426fb652f8)

7 years agoceph-volume util.arg_validators allow to pass a group as --data for lvm
Alfredo Deza [Thu, 12 Oct 2017 20:08:52 +0000 (16:08 -0400)]
ceph-volume util.arg_validators allow to pass a group as --data for lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 211c38f591945dee53196068533784c385bae102)

7 years agoceph-volume util.disk add utilities for is_device and is_partition
Alfredo Deza [Thu, 12 Oct 2017 20:08:16 +0000 (16:08 -0400)]
ceph-volume util.disk add utilities for is_device and is_partition

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b23fc72240ced158cf5335f6e815dbfd804dfab)

7 years agoceph-volume api.lvm udpate create_lv for bluestore
Alfredo Deza [Thu, 12 Oct 2017 20:05:22 +0000 (16:05 -0400)]
ceph-volume api.lvm udpate create_lv for bluestore

Require full size notation for LVM (e.g. 50G or 100M) and do not
"translate" tags by pre-fixing them with 'ceph'. That is already how the
rest of the API works, no need to make that translation in one place

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ac0d1555d3d9caeef9d4fc3e46bc11b7e437413b)

7 years agoceph-volume lvm.prepare update filestore workflow to use new helpers
Alfredo Deza [Thu, 12 Oct 2017 19:42:22 +0000 (15:42 -0400)]
ceph-volume lvm.prepare update filestore workflow to use new helpers

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1e8a79b8f4b9e6360e8e3ace580e46caeedca8c7)

7 years agoceph-volume lvm.prepare initial take on bluestore support
Alfredo Deza [Thu, 12 Oct 2017 18:19:07 +0000 (14:19 -0400)]
ceph-volume lvm.prepare initial take on bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e4fc3464af472a8dbdf049917eed73519ff82c3b)

7 years agoceph-volume lvm.common update help flags for bluestore usage
Alfredo Deza [Thu, 12 Oct 2017 18:16:25 +0000 (14:16 -0400)]
ceph-volume lvm.common update help flags for bluestore usage

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 17028f3a3c1f9394fa82dc6f2bbd596d96acb037)

7 years agoceph-volume api.lvm allow to create a vg
Alfredo Deza [Wed, 11 Oct 2017 18:30:41 +0000 (14:30 -0400)]
ceph-volume api.lvm allow to create a vg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 16d7feb2b5d5eb5fa0bf23a95b60623ae2e1676f)

7 years agoceph-volume lvm add block CLI flags for bluestore
Alfredo Deza [Tue, 10 Oct 2017 16:30:06 +0000 (12:30 -0400)]
ceph-volume lvm add block CLI flags for bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 63d2afbc605ac1531bc9f00a1891017bfd42fbce)

7 years agoceph-volume lvm make bluestore the default option
Alfredo Deza [Tue, 10 Oct 2017 16:15:14 +0000 (12:15 -0400)]
ceph-volume lvm make bluestore the default option

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dc3ced30747312dedd0e895a75ec15b1925ef1ea)

7 years agoMerge pull request #18433 from theanalyst/wip-21545-luminous
Abhishek L [Fri, 27 Oct 2017 13:11:00 +0000 (15:11 +0200)]
Merge pull request #18433 from theanalyst/wip-21545-luminous

luminous: rgw file write error

Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>
7 years agoMerge pull request #18587 from ceph/luminous-wip-volume-api-change
Andrew Schoen [Fri, 27 Oct 2017 13:03:16 +0000 (08:03 -0500)]
Merge pull request #18587 from ceph/luminous-wip-volume-api-change

luminous ceph-volume lvm api refactor/move

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoMerge pull request #18568 from liewegas/wip-fix-rop-leak-luminous-2
Sage Weil [Fri, 27 Oct 2017 11:39:58 +0000 (06:39 -0500)]
Merge pull request #18568 from liewegas/wip-fix-rop-leak-luminous-2

luminous: osd/PG: fix recovery op leak due to recovery preemption

Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoceph-volume create an api module to hold common api modules 18587/head
Alfredo Deza [Tue, 3 Oct 2017 20:13:33 +0000 (16:13 -0400)]
ceph-volume create an api module to hold common api modules

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f72368641d7f38b6395afb7ed70a0a9d0794a03a)

7 years agoceph-volume tests move lvm api tests into its new test module
Alfredo Deza [Wed, 4 Oct 2017 10:44:54 +0000 (06:44 -0400)]
ceph-volume tests move lvm api tests into its new test module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit eec0c8a5c27adc9b3fc5e999cbee8165bdb45736)

7 years agoceph-volume tests refactor lvm api import for test_listing
Alfredo Deza [Wed, 4 Oct 2017 10:43:36 +0000 (06:43 -0400)]
ceph-volume tests refactor lvm api import for test_listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 66fd41fd4b35e275196bfb94794f3fda9fd77f76)

7 years agoceph-volume tests refactor lvm api import for api tests
Alfredo Deza [Wed, 4 Oct 2017 10:43:18 +0000 (06:43 -0400)]
ceph-volume tests refactor lvm api import for api tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2a9a36bd24bddb35a7618127a0ec57f661a71566)

7 years agoceph-volume tests refactor lvm api import for activate tests
Alfredo Deza [Wed, 4 Oct 2017 10:42:47 +0000 (06:42 -0400)]
ceph-volume tests refactor lvm api import for activate tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d81aa8c112312f2fb49d94ce3512b3356b0b8814)

7 years agoceph-volume tests refactor lvm api import in conftest
Alfredo Deza [Wed, 4 Oct 2017 10:42:01 +0000 (06:42 -0400)]
ceph-volume tests refactor lvm api import in conftest

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a00aff7066fe6a679498ac8a02ec22ef29feced3)

7 years agoceph-volume devices.lvm refactor importing api to come from api/lvm.py
Alfredo Deza [Tue, 3 Oct 2017 20:17:08 +0000 (16:17 -0400)]
ceph-volume devices.lvm refactor importing api to come from api/lvm.py

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f4bee6cb42a017abce269ed3926deff04038a47e)

7 years agoceph-volume move lvm/api.py to api/lvm.py so disk can consume it
Alfredo Deza [Tue, 3 Oct 2017 20:14:16 +0000 (16:14 -0400)]
ceph-volume move lvm/api.py to api/lvm.py so disk can consume it

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bb72480d2bf44ff04ea93c98f4a4e59032a28896)

7 years agoMerge pull request #18498 from liewegas/wip-recovery-fixes-luminous
Sage Weil [Thu, 26 Oct 2017 21:52:28 +0000 (16:52 -0500)]
Merge pull request #18498 from liewegas/wip-recovery-fixes-luminous

luminous: osd: fix recovery priority and pg state on recovery->backfill transition

Reviewed-by: David Zafman <dzafman@redhat.com.
7 years agoMerge pull request #18567 from ceph/backport-wip-bz1491250
Andrew Schoen [Thu, 26 Oct 2017 21:50:20 +0000 (16:50 -0500)]
Merge pull request #18567 from ceph/backport-wip-bz1491250

luminous: ceph-volume lvm list

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoosd/PG: fix recovery op leak due to recovery preemption 18568/head
Sage Weil [Thu, 26 Oct 2017 21:23:41 +0000 (16:23 -0500)]
osd/PG: fix recovery op leak due to recovery preemption

This was fixed in master in a different patch, but are not yet ready to
backport the bits there that came before this.  For now, fix it
specifically for luminous.  We can either sort out the conflicts later
or revert this and backport the master parts conflict-free.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17805 from tchaikov/wip-luminous-pr-17508
Sage Weil [Thu, 26 Oct 2017 21:14:43 +0000 (16:14 -0500)]
Merge pull request #17805 from tchaikov/wip-luminous-pr-17508

luminous: mon: more aggressively convert crush rulesets -> distinct rules

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18364 from kmroz/wip-21544-luminous
Sage Weil [Thu, 26 Oct 2017 21:14:07 +0000 (16:14 -0500)]
Merge pull request #18364 from kmroz/wip-21544-luminous

luminous: mon: osd feature checks with 0 up osds

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18413 from kmroz/wip-21693-luminous
Sage Weil [Thu, 26 Oct 2017 21:13:31 +0000 (16:13 -0500)]
Merge pull request #18413 from kmroz/wip-21693-luminous

luminous: osd: additional protection for out-of-bounds EC reads

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #18529 from smithfarm/wip-21918-luminous
Sage Weil [Thu, 26 Oct 2017 21:11:42 +0000 (16:11 -0500)]
Merge pull request #18529 from smithfarm/wip-21918-luminous

luminous: Disable messenger logging (debug ms = 0/0) for clients unless overridden.

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18540 from liewegas/wip-pr-17894-luminous
Sage Weil [Thu, 26 Oct 2017 21:11:21 +0000 (16:11 -0500)]
Merge pull request #18540 from liewegas/wip-pr-17894-luminous

luminous: ceph.spec.in,debian/rules: change aio-max-nr to 1048576

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoceph-volume lvm add listing module to main 18567/head
Alfredo Deza [Wed, 13 Sep 2017 12:23:40 +0000 (08:23 -0400)]
ceph-volume lvm add listing module to main

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff72fd0ed52ef2d166007c7769c0140424982dc7)

7 years agodoc/ceph-volume add lvm/list to the toctree
Alfredo Deza [Tue, 3 Oct 2017 15:41:10 +0000 (11:41 -0400)]
doc/ceph-volume add lvm/list to the toctree

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d11c80c917f44b33077715b19564aeea30c78e42)

7 years agodoc/ceph-volume create the reference for the list sub-command
Alfredo Deza [Tue, 3 Oct 2017 15:36:06 +0000 (11:36 -0400)]
doc/ceph-volume create the reference for the list sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9a52939ef8fc4abe720d2525c59997f4f9760902)

7 years agodoc/ceph-volume add create and list to the index
Alfredo Deza [Tue, 3 Oct 2017 15:35:39 +0000 (11:35 -0400)]
doc/ceph-volume add create and list to the index

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ed5fa519aca1335039bfc32bc060af5181265517)

7 years agoceph-volume tests create a unit test module for listing features
Alfredo Deza [Tue, 3 Oct 2017 12:59:37 +0000 (08:59 -0400)]
ceph-volume tests create a unit test module for listing features

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2c2cf547b07f9510146b86d0746190b6d3fead3d)

7 years agoceph-volume tests create a small factory fixture
Alfredo Deza [Tue, 3 Oct 2017 12:59:01 +0000 (08:59 -0400)]
ceph-volume tests create a small factory fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5a0b17de81770ef3b851a1ff98232d93c78f49f9)

7 years agoceph-volume tests ensure Volume.as_dict gets populated correctly
Alfredo Deza [Mon, 2 Oct 2017 18:40:41 +0000 (14:40 -0400)]
ceph-volume tests ensure Volume.as_dict gets populated correctly

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c5b933a96d067fb8c7f753a67123e5662398f964)

7 years agoceph-volume lvm.listing initial take on listing command
Alfredo Deza [Mon, 2 Oct 2017 18:31:57 +0000 (14:31 -0400)]
ceph-volume lvm.listing initial take on listing command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2d80190a039798a637383513f292e6aab62cd3f8)

7 years agoceph-volume lvm.api be able to return a dictionary repr of a Volume object
Alfredo Deza [Mon, 2 Oct 2017 16:35:13 +0000 (12:35 -0400)]
ceph-volume lvm.api be able to return a dictionary repr of a Volume object

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0cae22750f396d1c1664875edb2742c50a41bffd)

7 years agoceph-volume process allows to fully mute terminal output on commands
Alfredo Deza [Mon, 2 Oct 2017 16:12:57 +0000 (12:12 -0400)]
ceph-volume process allows to fully mute terminal output on commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 233f64a4fd90a19c50d57b849ffaa6a25c835f34)

7 years agoceph-volume lvm.api use a helper to get an lv from an arg
Alfredo Deza [Thu, 14 Sep 2017 11:35:47 +0000 (07:35 -0400)]
ceph-volume lvm.api use a helper to get an lv from an arg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3c5bbbc5822f02a038f093647c5de58bef2fe8f5)

7 years agoMerge pull request #18456 from liewegas/wip-21882-luminous
Josh Durgin [Wed, 25 Oct 2017 18:45:38 +0000 (11:45 -0700)]
Merge pull request #18456 from liewegas/wip-21882-luminous

luminous: messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/suites/upgrade/jewel-x: make sure min compat client is hammer for straw2 18456/head
Sage Weil [Wed, 25 Oct 2017 12:00:14 +0000 (07:00 -0500)]
qa/suites/upgrade/jewel-x: make sure min compat client is hammer for straw2

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoceph.spec.in,debian/rules: change aio-max-nr to 1048576 18540/head
chenliuzhong [Tue, 24 Oct 2017 02:54:33 +0000 (10:54 +0800)]
ceph.spec.in,debian/rules: change aio-max-nr to 1048576

when osd is more than 14 in one host,it report error that aio is not enough.
As the default aio-max-nr is 65536, one OSD needs 4096 aios and other programs may use aios.
This patch change aio-max-nr to 1048576 when install ceph-osd rpm package and debian package

Signed-off-by: chenliuzhong <liuzhong.chen@easystack.cn>
(cherry picked from commit 36326dc7104fc2f20f19d51b6f618a029ba072d7)

7 years agocommon/common_init: disable ms subsystem log gathering for clients 18529/head
Jason Dillaman [Fri, 20 Oct 2017 02:13:36 +0000 (22:13 -0400)]
common/common_init: disable ms subsystem log gathering for clients

The log gathering causes large performance degradation to clients
with high message throughputs. This is hopefully a short-term
workaround until per-message logging can be replaced with an
efficient data recording system for post-incident analysis
use-cases.

Fixes: http://tracker.ceph.com/issues/21860
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a3a40413f7908b08c40dec4020034cca4a0c4798)

7 years agoMerge pull request #18503 from liewegas/wip-21878-luminous
Sage Weil [Wed, 25 Oct 2017 02:36:18 +0000 (21:36 -0500)]
Merge pull request #18503 from liewegas/wip-21878-luminous

luminous: os/bluestore/BlueFS: fix race with log flush during async log compaction

Reviewed-by: Varada Kari <varada.kari@gmail.com>
7 years agoMerge pull request #18501 from liewegas/wip-21766-luminous
Sage Weil [Wed, 25 Oct 2017 02:35:29 +0000 (21:35 -0500)]
Merge pull request #18501 from liewegas/wip-21766-luminous

luminous: os/bluestore: handle compressed extents in blob unsharing checks

7 years agoMerge pull request #18459 from liewegas/wip-pr-18457-luminous
Sage Weil [Wed, 25 Oct 2017 02:34:52 +0000 (21:34 -0500)]
Merge pull request #18459 from liewegas/wip-pr-18457-luminous

luminous: qa/suites/rados/rest/mgr-restful: whitelist more health

7 years agoosd/PrimaryLogPG: clear DEGRADED at recovery completion even if more backfill 18498/head
Sage Weil [Mon, 23 Oct 2017 22:11:59 +0000 (17:11 -0500)]
osd/PrimaryLogPG: clear DEGRADED at recovery completion even if more backfill

We may have log recovery *and* backfill to do, but cease to be degraded
as soon as the log recovery portion is done.  If that's the case, clear
the DEGRADED bit so that the PG state is not misleading.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e91fda79d3be771f46394312a6b72ab75660012)

7 years agoMerge pull request #18491 from liewegas/wip-21573-luminous
Kefu Chai [Tue, 24 Oct 2017 03:51:19 +0000 (11:51 +0800)]
Merge pull request #18491 from liewegas/wip-21573-luminous

luminous: buffer: fix ABI breakage by removing list _mempool member

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoos/bluestore/BlueFS: fix race with log flush during async log compaction 18503/head
Sage Weil [Fri, 20 Oct 2017 13:51:17 +0000 (08:51 -0500)]
os/bluestore/BlueFS: fix race with log flush during async log compaction

During async log compaction we rely on _flush-and_sync_log to update the
log_writer to jump_to.  However, if racing threads are also trying to flush
the log and manage to flush our new log events for us, then our flush will
turn into a no-op, and we won't update jump_to correctly at all.  This
results in a corrupted log size a bit later one.

Fix by ensuring that there are no in-progress flushes before we add our
log entries.  Also, add asserts to _flush_and_sync_log to make sure we
never bail out early if jump_to is set (which would indicate this or
another similar bug is still present).

Fixes: http://tracker.ceph.com/issues/21878
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4324c8bc7e66633035c15995e3f82ef91d3a5e8c)

7 years agoos/bluestore: handle compressed extents in blob unsharing checks 18501/head
Sage Weil [Wed, 11 Oct 2017 21:48:41 +0000 (16:48 -0500)]
os/bluestore: handle compressed extents in blob unsharing checks

If the blob is compressed, we aren't mapping to a range within
the allocated extents, but rather referencing the entire blob.

Fixes: http://tracker.ceph.com/issues/21766
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eb26cfbfbb2b9b9fe84dfe9c999e2d67fd2661c0)

7 years agoosd/PG: on recovery done, requeue for backfill
Sage Weil [Mon, 23 Oct 2017 03:46:00 +0000 (22:46 -0500)]
osd/PG: on recovery done, requeue for backfill

We were keeping our existing recovery reservation slot (with a high
priority) and going straight to waiting for backfill reservations on
the peers.  This is a problem because the reserver thinks we're doing
high priority work when we're actually doing lower-priority backfill.

Fix by closing out our recovery reservation and going to the
WaitLocalBackfillReserved state, where we'll re-request backfill at the
appropriate priority.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 09c50e2fe4fc3e8951a483e4b2693f0ffc008dcf)

7 years agoosd/PG: move local_reserver recovery cancel to Recovering state transition
Sage Weil [Mon, 23 Oct 2017 03:43:18 +0000 (22:43 -0500)]
osd/PG: move local_reserver recovery cancel to Recovering state transition

This is easier to follow than canceling the reservation in the next state.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dde007749bb960122f399b4cb8608a8c1ea4eb9c)

7 years agoqa/suites/upgrade/jewel-x/parallel: run some jewel after completed upgrade
Sage Weil [Sat, 21 Oct 2017 03:55:29 +0000 (22:55 -0500)]
qa/suites/upgrade/jewel-x/parallel: run some jewel after completed upgrade

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/upgrade/jewel-x/: set up compat weight-set after cluster upgrade
Sage Weil [Sat, 21 Oct 2017 03:52:45 +0000 (22:52 -0500)]
qa/suites/upgrade/jewel-x/: set up compat weight-set after cluster upgrade

...but before we run some client workloads.  This will hopefully ensure
that older clients will behave.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'
Sage Weil [Sat, 21 Oct 2017 18:03:17 +0000 (13:03 -0500)]
mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'

This is a common and recommended step that users should take once they
have clients that support straw2 (hammer!).  Let's make it easy.

This may result in a small amount of data movement, but usually it is
very little.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 928770513cbe3a244095e36dcb41c83c0724cfe9)

7 years agocrush/CrushWrapper: add bucket_set_alg()
Sage Weil [Sat, 21 Oct 2017 17:59:16 +0000 (12:59 -0500)]
crush/CrushWrapper: add bucket_set_alg()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit edc411ee4cb7b88a970ccfc9d29b3a3ba15d066c)

7 years agobuffer: remove list _mempool member 18491/head
Sage Weil [Thu, 19 Oct 2017 21:19:35 +0000 (16:19 -0500)]
buffer: remove list _mempool member

This broke the C++ ABI by changing the list structure size.  Also, it's
not necessary as we can infer the mempool by looking at the other list
contents.  We don't (currently) have a need to map an empty list to a
particular mempool and have that state stick.

Fixes: http://tracker.ceph.com/issues/21573
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b92d87d37e3892b0096cd728b46154aed1e2d86)

7 years agobuffer: allow mempool to be passed into raw* ctors and create methods
Sage Weil [Thu, 19 Oct 2017 21:09:10 +0000 (16:09 -0500)]
buffer: allow mempool to be passed into raw* ctors and create methods

This is more convenient, and also faster than initializing it in
buffer_anon and the immediately moving it elsewhere.

Drop the optionality of the alignment argument.

No users yet.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 481277b8c92368b8cac8fdafe1848720ec1035c2)

7 years ago/etc/sysconfig/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 17:05:39 +0000 (12:05 -0500)]
/etc/sysconfig/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 19efccef248c2d66fe3d7fdac8f56d455f2c7b27)

7 years agoetc/default/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 16:16:26 +0000 (11:16 -0500)]
etc/default/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fba2055b4e54128db1e9a83f27bb3526d65d4db7)

7 years agoqa/suites/rados/rest/mgr-restful: whitelist more health 18459/head
Sage Weil [Sat, 21 Oct 2017 04:14:36 +0000 (23:14 -0500)]
qa/suites/rados/rest/mgr-restful: whitelist more health

The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ddf3e9162da542af0c5f025957b8304e7359c924)

7 years agomon/OSDMonitor: improve crush map validation 17805/head
Sage Weil [Wed, 6 Sep 2017 02:46:48 +0000 (22:46 -0400)]
mon/OSDMonitor: improve crush map validation

- move into OSDMap method
- ensure that rules exist for each pool
- ensure pool type matches rule type
- ensure rule mask min/max size cover the pool size

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8f8f0b25058ff7319ec9d023f0eceb5832d0ecff)

7 years agomon: more forceful renumbering of legacy ruleset IDs
John Spray [Fri, 25 Aug 2017 10:06:21 +0000 (11:06 +0100)]
mon: more forceful renumbering of legacy ruleset IDs

Previously, the rules were only modified in the trivial case,
so we continued to potentially have CRUSH maps with the
legacy ruleset functionality in use.

In order to ultimately remove rulesets entirely, we need
to do this more aggressively, renumbering all the rules
and then updating any pools as needed.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 71d4b2bed54371657693cfb999ade44449be0efd)

Conflicts:
src/mon/OSDMonitor.cc: the check for multiple rules was removed
in master, but not in luminous. once we renumber the legacy ruleset IDs,
it's not need to check for and to warn the user at seeing the case where
1-to-n mapping from ruleset to rule IDs.

7 years agoosd: s/crush_ruleset_in_use/crush_rule_in_use/
John Spray [Fri, 25 Aug 2017 09:42:58 +0000 (10:42 +0100)]
osd: s/crush_ruleset_in_use/crush_rule_in_use/

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 675483ecca06686fd4c626bfea33748092aef0c6)

7 years agoMerge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag
Sage Weil [Sat, 21 Oct 2017 04:10:49 +0000 (23:10 -0500)]
Merge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag

luminous: osd/PrimaryLogPG: _delete_oid - fix incorrect 'legacy' flag

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18362 from kmroz/wip-21543-luminous
Sage Weil [Sat, 21 Oct 2017 04:10:06 +0000 (23:10 -0500)]
Merge pull request #18362 from kmroz/wip-21543-luminous

luminous: qa/tasks: prolong revive_osd() timeout to 6 min

7 years agoMerge pull request #18388 from tchaikov/wip-pr-18346-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:52 +0000 (23:09 -0500)]
Merge pull request #18388 from tchaikov/wip-pr-18346-luminous

os/filestore: print out the error if do_read_entry() fails

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #18398 from smithfarm/wip-21783-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:17 +0000 (23:09 -0500)]
Merge pull request #18398 from smithfarm/wip-21783-luminous

luminous: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18401 from liewegas/wip-bluestore-label-luminous
Sage Weil [Sat, 21 Oct 2017 04:08:58 +0000 (23:08 -0500)]
Merge pull request #18401 from liewegas/wip-bluestore-label-luminous

luminous: os/bluestore: ceph-bluestore-tool repair updates and prime-osd-dir support

7 years agomessages/MOSDMap: do compat reencode of crush map, too
Sage Weil [Sat, 21 Oct 2017 03:32:33 +0000 (22:32 -0500)]
messages/MOSDMap: do compat reencode of crush map, too

If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9eaafc66278d59447b29ce6cc378d0cd3ad14ad4)

7 years agomon/OSDMonitor: mon osd feature checks with 0 up osds 18364/head
Brad Hubbard [Wed, 20 Sep 2017 03:15:30 +0000 (13:15 +1000)]
mon/OSDMonitor: mon osd feature checks with 0 up osds

get_up_osd_features() returns 0 if no osds are up which causes feature
checks to fail.

Fixes: http://tracker.ceph.com/issues/21471
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 26ba0ba044846d18f9bdfdbee6e2a9c46882c742)

Conflicts:
src/mon/MonCommands.h
          - removed mimic reference
src/mon/OSDMonitor.cc
          - removed mimic-only command path

7 years agoosd/OSDMap: ignore xinfo if features == 0
Sage Weil [Tue, 19 Sep 2017 22:25:56 +0000 (18:25 -0400)]
osd/OSDMap: ignore xinfo if features == 0

Some old bug (e.g., http://tracker.ceph.com/issues/20751) could
result in an UP+EXISTS osd having features==0.  If that happens,
we shouldn't crash the mon, which (reasonably) does

   if (osdmap.get_epoch()) {
     if (osdmap.get_num_up_osds() > 0) {
       assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB);
       check_subs();
     }
   }

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8399833f3d221364a8ededb97cd4e1c5a258d924)

7 years agorgw_file: fix write error when the write offset overlaps. 18433/head
Yao Zongyou [Sun, 24 Sep 2017 14:11:22 +0000 (22:11 +0800)]
rgw_file: fix write error when the write offset overlaps.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit 872d73f945364002f0fa31762e6976db5b4b3c19)

7 years agoosd: additional protection for out-of-bounds EC reads 18413/head
Jason Dillaman [Mon, 2 Oct 2017 21:28:04 +0000 (17:28 -0400)]
osd: additional protection for out-of-bounds EC reads

Fixes: http://tracker.ceph.com/issues/21629
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 881f2a02aecef0cd6bf8ec3e5045b76c8441e832)

7 years agoMerge pull request #17692 from smithfarm/wip-21176-luminous
Sage Weil [Thu, 19 Oct 2017 21:43:39 +0000 (16:43 -0500)]
Merge pull request #17692 from smithfarm/wip-21176-luminous

build/ops: python-numpy-devel build dependency for SUSE

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #17730 from xiexingguo/wip-pr-17371
Sage Weil [Thu, 19 Oct 2017 21:42:45 +0000 (16:42 -0500)]
Merge pull request #17730 from xiexingguo/wip-pr-17371

luminous: mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17779 from smithfarm/wip-rh-74-luminous
Sage Weil [Thu, 19 Oct 2017 21:42:07 +0000 (16:42 -0500)]
Merge pull request #17779 from smithfarm/wip-rh-74-luminous

tests: CentOS 7.4 is now the latest

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
7 years agoMerge pull request #17863 from smithfarm/wip-21343-luminous
Sage Weil [Thu, 19 Oct 2017 21:41:34 +0000 (16:41 -0500)]
Merge pull request #17863 from smithfarm/wip-21343-luminous

luminous: mon: DNS SRV default service name not used anymore

7 years agoMerge pull request #17864 from smithfarm/wip-21438-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:45 +0000 (16:40 -0500)]
Merge pull request #17864 from smithfarm/wip-21438-luminous

luminous: core: Daemons(OSD, Mon...) exit abnormally at injectargs command

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #17866 from smithfarm/wip-21320-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:18 +0000 (16:40 -0500)]
Merge pull request #17866 from smithfarm/wip-21320-luminous

luminous: mgr: Quieten scary RuntimeError from restful module on startup

7 years agoMerge pull request #17951 from smithfarm/wip-21548-luminous
Sage Weil [Thu, 19 Oct 2017 21:39:53 +0000 (16:39 -0500)]
Merge pull request #17951 from smithfarm/wip-21548-luminous

luminous: tests: ceph_manager: bad AssertionError: failed to recover before timeout expired

7 years agoMerge pull request #18342 from xiexingguo/wip-pr-18318
Sage Weil [Thu, 19 Oct 2017 21:39:32 +0000 (16:39 -0500)]
Merge pull request #18342 from xiexingguo/wip-pr-18318

luminous: bluestore: using bluestore_compression_min_blob_size instead of bluestore_compression_max_blob_size to config comp_min_blob_size

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: fsync on _write_bdev_label 18401/head
Sage Weil [Mon, 16 Oct 2017 22:46:03 +0000 (17:46 -0500)]
os/bluestore: fsync on _write_bdev_label

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7aca8af6625156eeaa936629f83ede33d3bbe866)

7 years agoceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'
Sage Weil [Mon, 16 Oct 2017 22:45:54 +0000 (17:45 -0500)]
ceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4b468c989f9e8609e7f640cfa4385c454ae6dc17)

7 years agoos/bluestore: make _write_bdev_label static and public
Sage Weil [Mon, 16 Oct 2017 22:42:29 +0000 (17:42 -0500)]
os/bluestore: make _write_bdev_label static and public

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0b0550c967ae0d6ae49641c6859f0b63267e2cd9)

7 years agodoc/man/8/ceph-bluestore-tool: add man page
Sage Weil [Thu, 12 Oct 2017 16:40:11 +0000 (11:40 -0500)]
doc/man/8/ceph-bluestore-tool: add man page

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7b91e50dbd289e65f10cb98c5eea9f3171c2d1f1)

7 years agocommon/options: document 'key', 'keyfile', 'keyring'
Sage Weil [Fri, 13 Oct 2017 14:20:53 +0000 (09:20 -0500)]
common/options: document 'key', 'keyfile', 'keyring'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cdee8b26b9d93189dcd2875038361fb59ed21ba2)

7 years agoceph-bluestore-tool: fix show-label to use dev name as key
Sage Weil [Thu, 12 Oct 2017 18:54:06 +0000 (13:54 -0500)]
ceph-bluestore-tool: fix show-label to use dev name as key

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4785345ace7b675a0e1c34c457aad1103aab5a1d)

7 years agoceph-bluestore-tool: drop "action ..." print
Sage Weil [Thu, 12 Oct 2017 13:48:59 +0000 (08:48 -0500)]
ceph-bluestore-tool: drop "action ..." print

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5e486c2bd3414f4ee6519574d4d4584618144867)

7 years agoceph-bluestore-tool: only infer devs if devs.empty()
Sage Weil [Thu, 12 Oct 2017 13:48:46 +0000 (08:48 -0500)]
ceph-bluestore-tool: only infer devs if devs.empty()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2d217367af0df5e18394b1c6179d92db3c0b4448)

7 years agoceph-bluestore-tool: implement prime-osd-dir
Sage Weil [Wed, 11 Oct 2017 22:36:40 +0000 (17:36 -0500)]
ceph-bluestore-tool: implement prime-osd-dir

This populates an osd dir based on the metadata in a bluestore
device label.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 72ee7e29fb77f581a537b1ff2b274c9f3a7de53f)

7 years agoos/bluestore: keep meta items for extra bdev paths
Sage Weil [Wed, 11 Oct 2017 21:33:54 +0000 (16:33 -0500)]
os/bluestore: keep meta items for extra bdev paths

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d07fa5ad0772b4d979dc10695d230074d1f1e69)

7 years agovstart.sh: specify block files
Sage Weil [Wed, 11 Oct 2017 22:33:00 +0000 (17:33 -0500)]
vstart.sh: specify block files

This is mostly just to test ceph-bluestore-tool prime-osd-dir.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d4e0c46d89c2ebe6ae5f9af1c909a09daaea9687)