]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoqa/tasks: update mirror link for maven, the original mirror no longer exists 24038/head
Vasu Kulkarni [Thu, 6 Sep 2018 00:08:24 +0000 (17:08 -0700)]
qa/tasks: update mirror link for maven, the original mirror no longer exists

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 13e100259e3658fcfd2123474cb8ea8498ac3fb6)

6 years agoMerge pull request #23875 from smithfarm/wip-35079-mimic
Yuri Weinstein [Tue, 11 Sep 2018 16:04:28 +0000 (09:04 -0700)]
Merge pull request #23875 from smithfarm/wip-35079-mimic

mimic: mgr/dashboard: RestClient can't handle ProtocolError exceptions

Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #23966 from alfredodeza/mimic-rm35535
Andrew Schoen [Mon, 10 Sep 2018 14:28:58 +0000 (09:28 -0500)]
Merge pull request #23966 from alfredodeza/mimic-rm35535

mimic: ceph-volume: batch tests for mixed-type of devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #23744 from liewegas/wip-valgrind-distro-mimic
Yuri Weinstein [Sat, 8 Sep 2018 15:32:57 +0000 (08:32 -0700)]
Merge pull request #23744 from liewegas/wip-valgrind-distro-mimic

mimic: qa/suites/rados: move valgrind test to singleton-flat

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #23951 from smithfarm/wip-pglog-mimic
Yuri Weinstein [Sat, 8 Sep 2018 15:32:23 +0000 (08:32 -0700)]
Merge pull request #23951 from smithfarm/wip-pglog-mimic

mimic: osd/PrimaryLogPG: avoid dereferencing invalid complete_to

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #23885 from smithfarm/wip-24905-mimic
Yuri Weinstein [Fri, 7 Sep 2018 20:06:41 +0000 (13:06 -0700)]
Merge pull request #23885 from smithfarm/wip-24905-mimic

mimic: build/ops: mimic 13.2.0 doesn't build in Fedora rawhide

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23886 from smithfarm/wip-25119-mimic
Yuri Weinstein [Fri, 7 Sep 2018 20:05:48 +0000 (13:05 -0700)]
Merge pull request #23886 from smithfarm/wip-25119-mimic

mimic: tests: cluster [WRN] 25 slow requests in powercycle

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #23888 from smithfarm/wip-26933-mimic
Yuri Weinstein [Fri, 7 Sep 2018 20:05:05 +0000 (13:05 -0700)]
Merge pull request #23888 from smithfarm/wip-26933-mimic

mimic: osd: segv in OSDMap::calc_pg_upmaps from balancer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #23893 from pdvian/wip-32079-mimic
Yuri Weinstein [Fri, 7 Sep 2018 20:04:11 +0000 (13:04 -0700)]
Merge pull request #23893 from pdvian/wip-32079-mimic

mimic: test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #23874 from smithfarm/wip-25120-mimic
Nathan Cutler [Fri, 7 Sep 2018 19:56:01 +0000 (21:56 +0200)]
Merge pull request #23874 from smithfarm/wip-25120-mimic

mimic: mgr/dashboard: URL prefix is not working

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agomgr/dashboard: RestClient can't handle ProtocolError exceptions 23875/head
Volker Theile [Tue, 31 Jul 2018 08:43:06 +0000 (10:43 +0200)]
mgr/dashboard: RestClient can't handle ProtocolError exceptions

Fixes: https://tracker.ceph.com/issues/25190
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 446881f8fe26b5af5d535d72f912d16dbd01b500)

6 years agomgr/dashboard: Set timeout in RestClient calls
Volker Theile [Tue, 24 Jul 2018 09:52:29 +0000 (11:52 +0200)]
mgr/dashboard: Set timeout in RestClient calls

Set a default timeout of 45 seconds to all REST client calls. This can be customized via 'ceph dashboard set-rest-requests-timeout <seconds>'. Currently the REST client is only used by the RGW controller.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 23128391986bf89b29fad14e0f26e9cb6ecf5cc3)

Conflicts:
doc/mgr/dashboard.rst

6 years agoMerge pull request #23831 from pdvian/wip-27059-mimic
Nathan Cutler [Fri, 7 Sep 2018 08:27:03 +0000 (10:27 +0200)]
Merge pull request #23831 from pdvian/wip-27059-mimic

mimic : debian/rules: fix ceph-mgr .pyc files left behind

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoceph-volume lvm.batch use 'ceph' as the cluster name with filestore 23966/head
Alfredo Deza [Thu, 6 Sep 2018 19:19:01 +0000 (15:19 -0400)]
ceph-volume lvm.batch use 'ceph' as the cluster name with filestore

Custom cluster names are currently broken on ceph-volume, should get
addressed with http://tracker.ceph.com/issues/27210 which is out of
scope for these changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1481799a254eceb05258313aafd532e062ea0bc)

6 years agoceph-volume tests/functional update filestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:30 +0000 (11:23 -0400)]
ceph-volume tests/functional update filestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a096a016ccd0acc86ce816aba8ce95f6e3b68c11)

6 years agoceph-volume tests/functional update bluestore xenial test vars
Alfredo Deza [Thu, 6 Sep 2018 15:23:20 +0000 (11:23 -0400)]
ceph-volume tests/functional update bluestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89e52dd19792072919bef6ba2f2ab50d7b0ff4ed)

6 years agoceph-volume tests/functional update filestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:20 +0000 (11:18 -0400)]
ceph-volume tests/functional update filestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a5ec54207a02c4bb85dc8226890def18da45e2ff)

6 years agoceph-volume tests/functional update bluestore centos7 test vars
Alfredo Deza [Thu, 6 Sep 2018 15:18:05 +0000 (11:18 -0400)]
ceph-volume tests/functional update bluestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2549d5737283b39ecf5b0af98c73ffeb18033c36)

6 years agoceph-volume lvm.api use double -f flags when calling pvremove
Alfredo Deza [Thu, 6 Sep 2018 14:55:31 +0000 (10:55 -0400)]
ceph-volume lvm.api use double -f flags when calling pvremove

Fairly destructive, just like everything else when zapping a device.
This is required in the case of double UUIDs detected, something that
surfaced when testing with a loop device to create an nvme (the loop
device ends up with the same UUID as the nvme).

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1cbc407c095e0bc3e25a8ff1519b4909ec519cc)

6 years agoceph-volume tests/functional remove duplicate test playbooks
Alfredo Deza [Thu, 6 Sep 2018 14:26:05 +0000 (10:26 -0400)]
ceph-volume tests/functional remove duplicate test playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a008fe1385e4440f510eab97470a5d4e56a69402)

6 years agoceph-volume tests/functional setup playbook for nvme devices
Alfredo Deza [Thu, 6 Sep 2018 13:38:00 +0000 (09:38 -0400)]
ceph-volume tests/functional setup playbook for nvme devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71db509df0c4a14598010568f8bb078f6b2841ab)

6 years agoceph-volume tests/functional single (test) playbook to zap and redeploy batch operations
Alfredo Deza [Thu, 6 Sep 2018 13:34:11 +0000 (09:34 -0400)]
ceph-volume tests/functional single (test) playbook to zap and redeploy batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 23330c0107a9f9ee436338c112b1852fd36e34a3)

6 years agoceph-volume tests/functional create a noop playbook to allow symlinking always to...
Alfredo Deza [Thu, 6 Sep 2018 13:33:21 +0000 (09:33 -0400)]
ceph-volume tests/functional create a noop playbook to allow symlinking always to setup.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860e66083fe769714a10076fa101b94507e47169)

6 years agoceph-volume tests/functional add mixed-type scenarios to tox.ini
Alfredo Deza [Thu, 6 Sep 2018 13:32:00 +0000 (09:32 -0400)]
ceph-volume tests/functional add mixed-type scenarios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d5b90bb1fbf30c6e33dbf7e10a3d0ba053297ae)

6 years agoceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:54 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 799ce4508d9ed166ef6447daa21e0564262fbafe)

6 years agoceph-volume tests/functional add CentOS7 filestore mixed-type scenarios
Alfredo Deza [Wed, 5 Sep 2018 12:17:38 +0000 (08:17 -0400)]
ceph-volume tests/functional add CentOS7 filestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89bbcff659eed8f2cfa26fb93a0c12c7ff83e83b)

6 years agoMerge pull request #23854 from smithfarm/wip-35078-mimic
Yuri Weinstein [Thu, 6 Sep 2018 20:02:30 +0000 (13:02 -0700)]
Merge pull request #23854 from smithfarm/wip-35078-mimic

mimic: doc: broken bash example in bluestore migration

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #23868 from smithfarm/wip-27212-mimic
Nathan Cutler [Thu, 6 Sep 2018 11:14:50 +0000 (13:14 +0200)]
Merge pull request #23868 from smithfarm/wip-27212-mimic

mimic: rpm: should change ceph-mgr package depency from py-bcrypt to python2-bcrypt

Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoosd/PrimaryLogPG: avoid dereferencing invalid complete_to 23951/head
xie xingguo [Mon, 3 Sep 2018 07:37:36 +0000 (15:37 +0800)]
osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

6 years agoMerge pull request #23860 from smithfarm/wip-25144-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:52:00 +0000 (14:52 -0700)]
Merge pull request #23860 from smithfarm/wip-25144-mimic

mimic: mon: Automatically set expected_num_objects for new pools with >=100 PGs per OSD

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23869 from smithfarm/wip-27213-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:50:10 +0000 (14:50 -0700)]
Merge pull request #23869 from smithfarm/wip-27213-mimic

mimic: libradosstriper conditional compile

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23676 from ifed01/wip-ifed-fix-get_fragmentation
Sage Weil [Wed, 5 Sep 2018 21:43:33 +0000 (16:43 -0500)]
Merge pull request #23676 from ifed01/wip-ifed-fix-get_fragmentation

mimic: os/bluestore: fix assertion in StupidAllocator::get_fragmentation

6 years agoMerge pull request #23722 from pdvian/wip-26931-mimic
Sage Weil [Wed, 5 Sep 2018 21:43:09 +0000 (16:43 -0500)]
Merge pull request #23722 from pdvian/wip-26931-mimic

mimic: scrub livelock

6 years agoMerge pull request #23861 from smithfarm/wip-25176-mimic
Sage Weil [Wed, 5 Sep 2018 21:42:29 +0000 (16:42 -0500)]
Merge pull request #23861 from smithfarm/wip-25176-mimic

mimic: osd,mon: increase mon_max_pg_per_osd to 300

6 years agoMerge pull request #23863 from smithfarm/wip-25204-mimic
Sage Weil [Wed, 5 Sep 2018 21:42:07 +0000 (16:42 -0500)]
Merge pull request #23863 from smithfarm/wip-25204-mimic

mimic: rados python bindings use prval from stack

6 years agoMerge pull request #23870 from smithfarm/wip-32108-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:39:15 +0000 (14:39 -0700)]
Merge pull request #23870 from smithfarm/wip-32108-mimic

mimic: object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #23872 from smithfarm/wip-34532-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:38:24 +0000 (14:38 -0700)]
Merge pull request #23872 from smithfarm/wip-34532-mimic

mimic: core: force-create-pg broken

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #23873 from smithfarm/wip-35068-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:37:01 +0000 (14:37 -0700)]
Merge pull request #23873 from smithfarm/wip-35068-mimic

mimic: core: deep scrub cannot find the bitrot if the object is cached

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #23215 from b-ranto/wip-mgr-prom-bp-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:33:43 +0000 (14:33 -0700)]
Merge pull request #23215 from b-ranto/wip-mgr-prom-bp-mimic

[mimic] Sync the prometheus module

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #23403 from neha-ojha/wip-24989-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:32:13 +0000 (14:32 -0700)]
Merge pull request #23403 from neha-ojha/wip-24989-mimic

mimic: Limit pg log length during recovery/backfill so that we don't run out of memory

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #23748 from pdvian/wip-26944-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:30:35 +0000 (14:30 -0700)]
Merge pull request #23748 from pdvian/wip-26944-mimic

mimic: os/bluestore/BlueStore.cc: 1025: FAILED assert(buffer_bytes >= b->length) from ObjectStore/StoreTest.ColSplitTest2/2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #23772 from ifed01/wip-ifed-perf-reset-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:28:52 +0000 (14:28 -0700)]
Merge pull request #23772 from ifed01/wip-ifed-perf-reset-mimic

mimic: os/bluestore: bluestore_buffer_hit_bytes perf counter doesn't reset

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23782 from smithfarm/wip-32082-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:28:13 +0000 (14:28 -0700)]
Merge pull request #23782 from smithfarm/wip-32082-mimic

mimic: mgr balancer does not save optimized plan but latest

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #23865 from smithfarm/wip-26907-mimic
Yuri Weinstein [Wed, 5 Sep 2018 21:24:28 +0000 (14:24 -0700)]
Merge pull request #23865 from smithfarm/wip-26907-mimic

mimic: kv: MergeOperator name() returns string, and caller calls c_str() on the temporary

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23559 from dillaman/wip-25121-mimic
Yuri Weinstein [Wed, 5 Sep 2018 16:20:28 +0000 (09:20 -0700)]
Merge pull request #23559 from dillaman/wip-25121-mimic

mimic: rbd: improved trash snapshot namespace handling

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #23560 from dillaman/wip-24945-mimic
Yuri Weinstein [Wed, 5 Sep 2018 16:19:58 +0000 (09:19 -0700)]
Merge pull request #23560 from dillaman/wip-24945-mimic

mimic: librbd: validate data pool for self-managed snapshot support

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #23858 from smithfarm/wip-27060-mimic
Yuri Weinstein [Wed, 5 Sep 2018 16:19:24 +0000 (09:19 -0700)]
Merge pull request #23858 from smithfarm/wip-27060-mimic

mimic: run-rbd-unit-tests.sh test fails to finish in jenkin's make check run

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #23521 from pdvian/wip-26845-mimic
Yuri Weinstein [Wed, 5 Sep 2018 16:01:27 +0000 (09:01 -0700)]
Merge pull request #23521 from pdvian/wip-26845-mimic

mimic: rgw: Limit the number of lifecycle rules on one bucket.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
6 years agoMerge pull request #23686 from pdvian/wip-26921-mimic
Yuri Weinstein [Wed, 5 Sep 2018 16:00:05 +0000 (09:00 -0700)]
Merge pull request #23686 from pdvian/wip-26921-mimic

mimic: rgw: Fix log level of gc_iterate_entries

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
6 years agoMerge pull request #23855 from smithfarm/wip-26849-mimic
Yuri Weinstein [Wed, 5 Sep 2018 15:59:30 +0000 (08:59 -0700)]
Merge pull request #23855 from smithfarm/wip-26849-mimic

mimic: rgw: civetweb fails on urls with control characters

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
6 years agoMerge pull request #23771 from pdvian/wip-26946-mimic
Yuri Weinstein [Wed, 5 Sep 2018 15:56:53 +0000 (08:56 -0700)]
Merge pull request #23771 from pdvian/wip-26946-mimic

mimic: qa/rgw: patch keystone requirements.txt

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23856 from smithfarm/wip-26980-mimic
Yuri Weinstein [Wed, 5 Sep 2018 15:55:39 +0000 (08:55 -0700)]
Merge pull request #23856 from smithfarm/wip-26980-mimic

mimic: multisite: intermittent failures in test_bucket_sync_disable_enable

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23857 from smithfarm/wip-35070-mimic
Yuri Weinstein [Wed, 5 Sep 2018 15:55:10 +0000 (08:55 -0700)]
Merge pull request #23857 from smithfarm/wip-35070-mimic

mimic: cls/rgw: add rgw_usage_log_entry type to ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agotest: Use pids instead of jobspecs which were wrong 23893/head
David Zafman [Thu, 16 Aug 2018 21:30:09 +0000 (14:30 -0700)]
test: Use pids instead of jobspecs which were wrong

Fixes: http://tracker.ceph.com/issues/27056
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bc3317031057b5737710a095ce593135b62f9302)

7 years agoTestOSDMap: Test with negative upmap values 23888/head
Brad Hubbard [Fri, 3 Aug 2018 06:42:57 +0000 (16:42 +1000)]
TestOSDMap: Test with negative upmap values

Check we can handle negative pg_upmap and pg_upmap_items values.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f76c889c2cb02539f5c5bdfc7bd5ab8f506bad86)

7 years agoosd/OSDMap: Disallow negative values in _apply_upmap
Brad Hubbard [Fri, 3 Aug 2018 06:12:49 +0000 (16:12 +1000)]
osd/OSDMap: Disallow negative values in _apply_upmap

A negative pg_upmap or pg_upmap_item value can cause a segfault so
ignore them.

Fixes: http://tracker.ceph.com/issues/22056
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 689e1cfba74e9866d411d5d21e0cf0b28343519c)

7 years agoqa/tasks/thrashosds-health.yaml: whitelist slow requests 23886/head
Neha Ojha [Wed, 25 Jul 2018 17:20:12 +0000 (10:20 -0700)]
qa/tasks/thrashosds-health.yaml: whitelist slow requests

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c0d9f7177c47a2b613fb1a3e9fa56eb90b36973a)

7 years agocmake: do not depend on ${DPDK_LIBRARIES} if not using bundled dpdk 23885/head
Kefu Chai [Sun, 8 Jul 2018 10:21:37 +0000 (18:21 +0800)]
cmake: do not depend on ${DPDK_LIBRARIES} if not using bundled dpdk

${DPDK_LIBRARIES} will be link flags instead of library targets if
libdpdk is not built using the bundled source.

Fixes: http://tracker.ceph.com/issues/24449
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1c41870184a3857d94ccf2c77958e2f0aad8dad8)

7 years agocmake: add option WITH_LIBRADOSSTRIPER 23869/head
Kefu Chai [Fri, 24 Aug 2018 14:33:01 +0000 (22:33 +0800)]
cmake: add option WITH_LIBRADOSSTRIPER

otherwise this cmake variable is never defined, hence all tests
requiring libstriper are broken.

this is a regresssion introduced by 5513a90c62 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e17900d552aef1ac96e5d1d60ec3ff179ceb38e9)

7 years agotools/rados: include acconfig.h for picking up WITH_LIBRADOSSTRIPER
Kefu Chai [Fri, 24 Aug 2018 16:43:29 +0000 (00:43 +0800)]
tools/rados: include acconfig.h for picking up WITH_LIBRADOSSTRIPER

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9497775cb4426f319987cf2a03ca171281e4e6c7)

7 years agomgr/dashboard: Fix cherrypy static content URL prefix config 23874/head
Ricardo Marques [Mon, 23 Jul 2018 14:43:04 +0000 (15:43 +0100)]
mgr/dashboard: Fix cherrypy static content URL prefix config

Fixes: https://tracker.ceph.com/issues/25067
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit a36eff1e51928777f9efda62da723902257f9d48)

7 years agoos/bluestore: fix deep-scrub operation againest disk silent errors 23873/head
Xiaoguang Wang [Thu, 30 Aug 2018 02:26:41 +0000 (10:26 +0800)]
os/bluestore: fix deep-scrub operation againest disk silent errors

Say a object who has data caches, but in a while later, caches' underlying
physical device has silent disk erros accidentally, then caches and physical
data are not same. In such case, deep-scrub operation still tries to read
caches firstly and won't do crc checksum, then deep-scrub won't find such
data corruptions timely.

Here introduce a new flag 'CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE' which tells
deep-scrub to bypass object caches. Note that we only bypass cache who is in
STATE_CLEAN state. For STATE_WRITING caches, currently they are not written
to physical device, so deep-scrub operation can not read physical device and
can read these dirty caches safely. Once they are in STATE_CLEAN state(or not
added to bluestore cache), next round deep-scurb can check them correctly.

As to above discussions, I refactor BlueStore::BufferSpace::read sightly,
adding a new 'flags' argument, whose value will be 0 or:
     enum {
       BYPASS_CLEAN_CACHE = 0x1,     // bypass clean cache
     };

flags 0: normal read, do not bypass clean or dirty cache
flags BYPASS_CLEAN_CACHE: bypass clean cache, currently only for deep-scrube
                        operation

Test:
   I deliberately corrupt a object with cache, with this patch, deep-scrub
   can find data error very timely.

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@easystack.cn>
(cherry picked from commit a7f1af25dd2ba88a322ed21828f073a277b09d02)

7 years agoos: Use ceph_assert for asserts
Adam C. Emerson [Thu, 23 Aug 2018 15:17:42 +0000 (11:17 -0400)]
os: Use ceph_assert for asserts

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit c3321f7633670ab84899d85da289109f3b56e886)

Conflicts:
src/os/bluestore/BlueFS.cc
src/os/bluestore/BlueStore.cc
src/os/bluestore/KernelDevice.cc
src/os/bluestore/StupidAllocator.cc
src/os/bluestore/bluestore_types.h
src/os/bluestore/fastbmap_allocator_impl.cc
src/os/bluestore/fastbmap_allocator_impl.h
src/os/filestore/FDCache.h

7 years agocore: add missing flag name for CEPH_OSD_OP_FLAG_WITH_REFERENCE
Xiaoguang Wang [Thu, 30 Aug 2018 02:07:45 +0000 (10:07 +0800)]
core: add missing flag name for CEPH_OSD_OP_FLAG_WITH_REFERENCE

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@easystack.cn>
(cherry picked from commit 2c862ce6e92bd28ef9e71fdf6a1f88a6e0173043)

7 years agomon/MonCommands: fix 'osd force-create-pg' 23872/head
Sage Weil [Tue, 14 Aug 2018 15:54:22 +0000 (10:54 -0500)]
mon/MonCommands: fix 'osd force-create-pg'

Fixes: 7797ed67d2f9140b7eb9f182b06d04233e1e309c
Fixes: http://tracker.ceph.com/issues/26940
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 84843422f402117f10a779b070bd19f30bd3b514)

7 years agotest: Update tests for error message changes 23870/head
David Zafman [Thu, 16 Aug 2018 22:31:53 +0000 (15:31 -0700)]
test: Update tests for error message changes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 687f63e59970433bfb53c17d4128a3d042a0c9c6)

7 years agoosd: Use colon to separate error message from other info
David Zafman [Thu, 16 Aug 2018 22:24:11 +0000 (15:24 -0700)]
osd: Use colon to separate error message from other info

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bdd32cc84787892b3005f2644b5ad9f8ac7e772d)

7 years agoqa: Add new message to whitelist for scrub/repair tests
David Zafman [Mon, 6 Aug 2018 17:37:04 +0000 (10:37 -0700)]
qa: Add new message to whitelist for scrub/repair tests

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b40784290f3b600941e0d97372fddee5c85ec20c)

7 years agotest: Verify cluster logging of scrub error messages
David Zafman [Fri, 3 Aug 2018 20:47:30 +0000 (13:47 -0700)]
test: Verify cluster logging of scrub error messages

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 58c4d322032d04b0d55f627ce42531ca3edab3e9)

7 years agoosd: Log shard errors in be_select_auth_object() as other errors
David Zafman [Wed, 1 Aug 2018 22:13:40 +0000 (15:13 -0700)]
osd: Log shard errors in be_select_auth_object() as other errors

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cadf727258ea8b3a52ae9122b42a0b4f6914e177)

Conflicts:
src/osd/PGBackend.cc

- mimic still has oi_prio

7 years agoosd: Move object error check out of be_select_auth_object()
David Zafman [Wed, 1 Aug 2018 00:53:17 +0000 (17:53 -0700)]
osd: Move object error check out of be_select_auth_object()

Fixes: http://tracker.ceph.com/issues/25108
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c789fd1899f697a994a28e45a6f9262c6721d77d)

Conflicts:
src/osd/PGBackend.cc

- mimic still has auth_prio

7 years agopackage: modify ceph.spec.in to support libradosstriper conditional compilation
Jesse Williamson [Mon, 13 Aug 2018 22:39:58 +0000 (15:39 -0700)]
package: modify ceph.spec.in to support libradosstriper conditional compilation

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
(cherry picked from commit 1a44357f5c51b61dc4f6735b4b75715d66baa73c)

Conflicts:
ceph.spec.in

- no cephfs shell in mimic

7 years agotools: make rados.cc compatible with conditionally-compiled striper
Jesse Williamson [Fri, 11 May 2018 00:53:22 +0000 (17:53 -0700)]
tools: make rados.cc compatible with conditionally-compiled striper

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
(cherry picked from commit 6e4db7b0a41f8d56c8725ea27352e85a13273888)

Conflicts:
src/tools/rados/rados.cc

825c7b9729b2410a116e123d4b21210c646704d4 not backported to mimic

7 years agobuild: add top-level libraddosstriper conditional
Jesse Williamson [Thu, 10 May 2018 23:20:42 +0000 (16:20 -0700)]
build: add top-level libraddosstriper conditional

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
(cherry picked from commit 5513a90c62f506d251dfebb2c3a871480c36337a)

7 years agorpm: change ceph-mgr package depency from py-bcrypt to python2-bcrypt 23868/head
Konstantin Sakhinov [Thu, 16 Aug 2018 19:38:17 +0000 (23:38 +0400)]
rpm: change ceph-mgr package depency from py-bcrypt to python2-bcrypt

Signed-off-by: Konstantin Sakhinov <sakhinov@gmail.com>
(cherry picked from commit 6af10cae66bfd2fd31ead1ad5ef187a6a4d25acd)

7 years agokv/KeyValueDB: return const char* from MergeOperator::name() 23865/head
Sage Weil [Tue, 7 Aug 2018 22:02:56 +0000 (17:02 -0500)]
kv/KeyValueDB: return const char* from MergeOperator::name()

This avoids a c_str() on a temporary string later down the line, which
avoids a use-after-free.

Fixes: http://tracker.ceph.com/issues/26875
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e9be5dabf256c18a5575d1394e481e4f21ece97d)

7 years agopybind/rados/rados: do not pass prval from stack 23863/head
Sage Weil [Mon, 30 Jul 2018 19:18:07 +0000 (14:18 -0500)]
pybind/rados/rados: do not pass prval from stack

The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8e36f18cdeaa2088a6ce4aaad61b76283e777270)

7 years agoosd,mon: increase mon_max_pg_per_osd to 250 23861/head
Neha Ojha [Thu, 26 Jul 2018 00:25:35 +0000 (17:25 -0700)]
osd,mon: increase mon_max_pg_per_osd to 250

https://bugzilla.redhat.com/show_bug.cgi?id=1603615 indicates
a case when pg calc conflicts with mon_max_pg_per_osd, and does not
allow pool creation when this limit is 200. Hence, increase this limit
to avoid this.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f7911de61bca7c7f1ef007eb229ee845923c9751)

7 years agomon/OSDMonitor: Warn if missing expected_num_objects 23860/head
Douglas Fuller [Fri, 29 Jun 2018 17:55:31 +0000 (13:55 -0400)]
mon/OSDMonitor: Warn if missing expected_num_objects

When creating a pool on filestore, warn if the user appears to be
creating a pool to store a large number of objects but omitted the
expected_num_objects parameter. Create the pool anyway.

Fixes: http://tracker.ceph.com/issues/24687
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 69fb2293c4d38012e7c4781aaa39a47596125bbb)

7 years agomon/OSDMonitor: Warn when expected_num_objects will have no effect
Douglas Fuller [Thu, 28 Jun 2018 15:21:38 +0000 (11:21 -0400)]
mon/OSDMonitor: Warn when expected_num_objects will have no effect

The expected_num_objects argument to ceph osd pool create is
only effective on filestore pools when merging is disabled
(filestore_merge_threshold < 0). Warn and disallow pool creation
in this situation.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 4c108a50e5f74a56965d49687a8c817f4a5ce42b)

7 years agolibrbd: potential race on image create request complete 23858/head
Mykola Golub [Sat, 18 Aug 2018 16:24:55 +0000 (19:24 +0300)]
librbd: potential race on image create request complete

Fixes: http://tracker.ceph.com/issues/24910
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d9e5cfaf4a8cd090362892805e837f777b220474)

7 years agocls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder 23857/head
Vaibhav Bhembre [Thu, 31 May 2018 14:35:19 +0000 (10:35 -0400)]
cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Signed-off-by: Vaibhav Bhembre vaibhav@digitalocean.com
(cherry picked from commit a503cec93853235f2f94dbc53eb307618f5a437a)

7 years agorgw: bucket sync transitions back to StateInit on OP_SYNCSTOP 23856/head
Casey Bodley [Tue, 14 Aug 2018 18:06:40 +0000 (14:06 -0400)]
rgw: bucket sync transitions back to StateInit on OP_SYNCSTOP

the handling for OP_SYNCSTOP->OP_RESYNC is incorrect because it's not
safe to continue incremental sync after SYNCSTOP. any changes between
SYNCSTOP and RESYNC will not be written to the bilog, so incremental
sync cannot sync them

SYNCSTOP now transitions back to StateInit, where
RGWInitBucketShardSyncStatusCoroutine will query the remote bilog info
to see if it's still disabled. if so, it deletes the sync status object
and finishes. otherwise, StateInit will transition back to StateFull to
resync the bucket

Fixes: http://tracker.ceph.com/issues/26895
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d7725a091890cd8e52938c070cc15da3fe57a9e7)

7 years agorgw: bucket sync crs mutate rgw_bucket_shard_sync_info in place
Casey Bodley [Tue, 14 Aug 2018 17:39:32 +0000 (13:39 -0400)]
rgw: bucket sync crs mutate rgw_bucket_shard_sync_info in place

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6c6917bca167819e890d1df3aa52b31a8f070e26)

7 years agorgw: return errors from RGWInitBucketShardSyncStatusCoroutine
Casey Bodley [Tue, 14 Aug 2018 17:06:50 +0000 (13:06 -0400)]
rgw: return errors from RGWInitBucketShardSyncStatusCoroutine

return ENOENT so caller can detect whether sync is stopped

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 72ac0aa1bb5bdffb06b1df8469206b3967a59aa0)

7 years agocivetweb: update for url validation fixes 23855/head
Abhishek Lekshmanan [Thu, 17 May 2018 10:00:54 +0000 (12:00 +0200)]
civetweb: update for url validation fixes

The new civetweb 1.10 version in mimic and later is strict on control characters
being url encoded, making url validation more relaxed and passing these through
to rgw where the requisite url validation is done.

Fixes: http://tracker.ceph.com/issues/24158
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 810c1fa8d3a253745d07c4c740296589fa43811a)

7 years agodoc/rados update invalid bash on bluestore migration 23854/head
Alfredo Deza [Wed, 29 Aug 2018 14:26:11 +0000 (07:26 -0700)]
doc/rados update invalid bash on bluestore migration

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 28bbcb2bafc166afc5733029b5d3c770e21df73b)

7 years agoMerge pull request #23607 from joscollin/wip-26916-mimic
Nathan Cutler [Fri, 31 Aug 2018 17:47:18 +0000 (19:47 +0200)]
Merge pull request #23607 from joscollin/wip-26916-mimic

mimic: doc: Fix broken urls

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agodebian/rules: fix ceph-mgr .pyc files left behind 23831/head
Dan Mick [Thu, 16 Aug 2018 01:32:22 +0000 (18:32 -0700)]
debian/rules: fix ceph-mgr .pyc files left behind

Add second dh_python2 call with the "private" dir /usr/lib/ceph/mgr

Fixes: http://tracker.ceph.com/issues/26883
Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit b221877462c021ebaf4ab03e37aefb938d86875d)

7 years agoosd/PrimaryLogPG.cc: limit trimming at can_rollback_to 23403/head
Neha Ojha [Thu, 16 Aug 2018 18:48:19 +0000 (11:48 -0700)]
osd/PrimaryLogPG.cc: limit trimming at can_rollback_to

This change is motivated by the failures seen in the multimds suite,
where we hit assert(s <= can_rollback_to), while trimming the log in ec
pools.

This is due to the fact that we had removed limits on the trim_to value to
address https://tracker.ceph.com/issues/23979.

But, seems that this could be dangerous for ec pools. So, keep the
can_rollback_to limit, while calculating the trim_to value.

Fixes: http://tracker.ceph.com/issues/21416
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b5c6b88d444e2173e716fe4890717873c8dc8e5)

7 years agoMerge pull request #23826 from alfredodeza/mimic-rm34309
Andrew Schoen [Thu, 30 Aug 2018 14:52:00 +0000 (09:52 -0500)]
Merge pull request #23826 from alfredodeza/mimic-rm34309

mimic: ceph-volume: update batch documentation to explain filestore strategies

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agodoc/man/ceph-volume uncomment flags that are now fully supported in batch 23826/head
Alfredo Deza [Tue, 28 Aug 2018 20:34:25 +0000 (13:34 -0700)]
doc/man/ceph-volume uncomment flags that are now fully supported in batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f8db9faf9836328b920133c5c9c47a6ffacea443)

7 years agodoc/ceph-volume add filestore docs for batch
Alfredo Deza [Tue, 28 Aug 2018 20:32:19 +0000 (13:32 -0700)]
doc/ceph-volume add filestore docs for batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 188f4ae80fefcdd8d95d719a88d8422b2c120f0c)

7 years agodoc/ceph-volume uncomment section of newly supported batch operations
Alfredo Deza [Tue, 28 Aug 2018 20:22:20 +0000 (13:22 -0700)]
doc/ceph-volume uncomment section of newly supported batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 02652343c8501a144c5ae2c012f335e8d0f236cf)

7 years agoMerge pull request #23812 from ceph/backport-mimic-23788
Alfredo Deza [Wed, 29 Aug 2018 20:47:51 +0000 (13:47 -0700)]
Merge pull request #23812 from ceph/backport-mimic-23788

mimic: ceph-volume: tests.functional inherit SSH_ARGS from ansible

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume tests.functional use Debian ansible os family in deploy.yml 23812/head
Alfredo Deza [Wed, 29 Aug 2018 17:32:55 +0000 (10:32 -0700)]
ceph-volume tests.functional use Debian ansible os family in deploy.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7a076f5e4f58bc3b346cde306ba949aabbb9b743)

7 years agoceph-volume lvm.batch use the abspath attribute for Device objects in bluestore
Alfredo Deza [Wed, 29 Aug 2018 16:56:21 +0000 (09:56 -0700)]
ceph-volume lvm.batch use the abspath attribute for Device objects in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b2a62c2b13d3e079f8020865b7c41f0f6c4b5d71)

7 years agoceph-volume lvm.batch remove usage of non-existent method
Alfredo Deza [Wed, 29 Aug 2018 16:55:31 +0000 (09:55 -0700)]
ceph-volume lvm.batch remove usage of non-existent method

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6f3094449f46d10260c0a505b39571c35235292e)

7 years agoceph-volume tests.functional inherit SSH_ARGS from ansible with synchronize module
Alfredo Deza [Tue, 28 Aug 2018 22:34:58 +0000 (15:34 -0700)]
ceph-volume tests.functional inherit SSH_ARGS from ansible with synchronize module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8e6269336f6e177f415a51e9fedd36244014215a)

7 years agoMerge pull request #23810 from alfredodeza/mimic-rm34310
Andrew Schoen [Wed, 29 Aug 2018 19:38:41 +0000 (14:38 -0500)]
Merge pull request #23810 from alfredodeza/mimic-rm34310

mimic: ceph-volume lvm.batch remove non-existent sys_api property

Reviewed-by: Andrew Schoen <aschoen@redhat.com>