]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #27321 from liewegas/wip-valgrind-mimic
Sage Weil [Wed, 3 Apr 2019 14:56:58 +0000 (09:56 -0500)]
Merge pull request #27321 from liewegas/wip-valgrind-mimic

mimic: qa/valgrind: backport of latest valgrind suppressions

6 years agoqa/valgrind: backport of latest valgrind suppressions 27321/head
Sage Weil [Tue, 2 Apr 2019 15:56:07 +0000 (10:56 -0500)]
qa/valgrind: backport of latest valgrind suppressions

in the correct location

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26983 from ifed01/wip-ifed-bmap-alloc-mimic
Yuri Weinstein [Mon, 1 Apr 2019 13:52:44 +0000 (06:52 -0700)]
Merge pull request #26983 from ifed01/wip-ifed-bmap-alloc-mimic

mimic: os/bluestore: backport new bitmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26961 from pdvian/wip-38666-mimic
Yuri Weinstein [Wed, 27 Mar 2019 19:58:29 +0000 (12:58 -0700)]
Merge pull request #26961 from pdvian/wip-38666-mimic

mimic: qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27053 from rhcs-dashboard/38284-mimic-show-io-stats
Lenz Grimmer [Wed, 27 Mar 2019 10:04:09 +0000 (11:04 +0100)]
Merge pull request #27053 from rhcs-dashboard/38284-mimic-show-io-stats

mimic: mgr/dashboard: show I/O stats in Pool list

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoqa/suites: fix bluestore links 26983/head
Sage Weil [Wed, 5 Dec 2018 16:27:53 +0000 (10:27 -0600)]
qa/suites: fix bluestore links

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 269910fc8b2bdf64b98534ce525318e108aa9409)

6 years agoMerge pull request #27081 from dzafman/wip-38275
David Zafman [Mon, 25 Mar 2019 15:04:58 +0000 (08:04 -0700)]
Merge pull request #27081 from dzafman/wip-38275

mimic: Fix recovery and backfill priority handling

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27032 from cbodley/wip-38797
Nathan Cutler [Fri, 22 Mar 2019 13:23:10 +0000 (14:23 +0100)]
Merge pull request #27032 from cbodley/wip-38797

mimic: doc/rgw: document placement target configuration

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agomgr/dashboard: mimic fix: show I/O stats in Pool list 27053/head
alfonsomthd [Thu, 21 Mar 2019 11:58:55 +0000 (12:58 +0100)]
mgr/dashboard: mimic fix: show I/O stats in Pool list

* Applied same fix as in nautilus.
* Adapted Pool list columns to avoid constant resizing.
* Added 'decimals' argument to dimless pipe in order for
  stats to fit in Pool list columns.

Fixes: https://tracker.ceph.com/issues/38284
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoosd: Add OSD_BACKFILL_PRIORITY_FORCED with lower priority 27081/head
David Zafman [Fri, 1 Feb 2019 07:04:03 +0000 (23:04 -0800)]
osd: Add OSD_BACKFILL_PRIORITY_FORCED with lower priority
than OSD_RECOVERY_PRIORITY_FORCED

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d088ffb09f46c6207cfedd2cdab9f074197af4bb)

6 years agotest: Add scripts to test backfill/recovery priority handling
David Zafman [Sat, 26 Jan 2019 19:16:17 +0000 (11:16 -0800)]
test: Add scripts to test backfill/recovery priority handling

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ca5cf14fa83ee89260688d467257bb3d81efdd36)

6 years agoosd AsyncReserver: Update priority in AsyncReserver for force and cancel-force
David Zafman [Sat, 26 Jan 2019 00:04:11 +0000 (16:04 -0800)]
osd AsyncReserver: Update priority in AsyncReserver for force and cancel-force

Fixes: http://tracker.ceph.com/issues/38041
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 11097f1235265a3eb753adc7e6d2818e5902e39e)

Backport change:
src/common/AsyncReserver.h (Mimic locking in update_priority())

6 years agotest: Add option to wait_for_clean() to execute at every sleep
David Zafman [Sat, 26 Jan 2019 00:03:30 +0000 (16:03 -0800)]
test: Add option to wait_for_clean() to execute at every sleep

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 70b51362084d2ae5f2a6e9e5856f17452c53e3f3)

6 years agoMerge pull request #26578 from badone/wip-mimic-test-dup_index-trim
Brad Hubbard [Tue, 19 Mar 2019 22:39:30 +0000 (08:39 +1000)]
Merge pull request #26578 from badone/wip-mimic-test-dup_index-trim

mimic: test: Verify a log trim trims the dup_index

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26583 from pdvian/wip-38398-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:38:52 +0000 (13:38 -0700)]
Merge pull request #26583 from pdvian/wip-38398-mimic

mimic: common: ceph_timer: stop timer's thread when it is suspended

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26612 from badone/wip-mimic-drop-GIL-in-get_config
Yuri Weinstein [Tue, 19 Mar 2019 20:38:26 +0000 (13:38 -0700)]
Merge pull request #26612 from badone/wip-mimic-drop-GIL-in-get_config

mimic: mgr: drop GIL in get_config

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26707 from pdvian/wip-38505-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:38:00 +0000 (13:38 -0700)]
Merge pull request #26707 from pdvian/wip-38505-mimic

mimic: rbd: API list_images() Segmentation fault

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #26944 from ashishkumsingh/wip-38725-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:37:22 +0000 (13:37 -0700)]
Merge pull request #26944 from ashishkumsingh/wip-38725-mimic

mimic: mgr/dashboard: Search broken for entries with null values

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge pull request #26517 from yehudasa/wip-rgw-es-fixes-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:34:44 +0000 (13:34 -0700)]
Merge pull request #26517 from yehudasa/wip-rgw-es-fixes-mimic

mimic: rgw: multiple es related fixes and improvements

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26687 from pdvian/wip-38502-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:34:11 +0000 (13:34 -0700)]
Merge pull request #26687 from pdvian/wip-38502-mimic

mimic: rgw: rgw_file: use correct secret key to check auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26664 from pdvian/wip-38415-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:31:09 +0000 (13:31 -0700)]
Merge pull request #26664 from pdvian/wip-38415-mimic

mimic: rgw: bug in versioning concurrent, list and get have consistency issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26945 from ashishkumsingh/wip-38726-mimic
Yuri Weinstein [Tue, 19 Mar 2019 20:30:28 +0000 (13:30 -0700)]
Merge pull request #26945 from ashishkumsingh/wip-38726-mimic

mimic: rgw: bucket limit check misbehaves for > max-entries buckets (usually 1000)

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #26702 from ceph/backport-mimic-26685
Alfredo Deza [Tue, 19 Mar 2019 04:32:16 +0000 (00:32 -0400)]
Merge pull request #26702 from ceph/backport-mimic-26685

mimic: ceph-volume: use our own testinfra suite for functional testing

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agodoc/rgw: document placement target configuration 27032/head
Casey Bodley [Fri, 15 Mar 2019 18:35:13 +0000 (14:35 -0400)]
doc/rgw: document placement target configuration

Fixes: http://tracker.ceph.com/issues/24508
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c95585b1984e60708addee436a774d0c92ce57a6)

6 years agoMerge pull request #27014 from badone/wip-mimic-CA-mods
Brad Hubbard [Sun, 17 Mar 2019 21:08:35 +0000 (07:08 +1000)]
Merge pull request #27014 from badone/wip-mimic-CA-mods

mimic: qa/ceph-ansible: modifications for mimic branch

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa/objectstore: bluestore -> bluestore-{bitmap,stupid}
Sage Weil [Fri, 30 Nov 2018 15:55:38 +0000 (09:55 -0600)]
qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

The new default is bitmap, so we were testing bitmap twice.  Instead,
explicitly call out stupid and bitmap cases so a future default change
won't break coverage.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f390704b4616c837352e7d6a8df91526cb047f0d)

6 years agoos/bluestore: fix minor issues in bitmap logging output
Igor Fedotov [Mon, 25 Jun 2018 14:49:49 +0000 (17:49 +0300)]
os/bluestore: fix minor issues in bitmap logging output

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dc4b5fc5afcea4be9e3b4c91dcc9dcce38d4e906)

6 years agoos/bluestore: fix overlappedd allocation returned by bitmap allocator
Igor Fedotov [Mon, 25 Jun 2018 14:42:41 +0000 (17:42 +0300)]
os/bluestore: fix overlappedd allocation returned by bitmap allocator

Fixes: https://tracker.ceph.com/issues/24598
Signed-off-by: Igor Fedotov <ifedotov@suse.som>
(cherry picked from commit 539af4f27e773cb21d187b75086c7c9498dd7749)

6 years agoos/bluestore: provide a rough estimate for bitmap allocator
Igor Fedotov [Fri, 15 Jun 2018 14:04:57 +0000 (17:04 +0300)]
os/bluestore: provide a rough estimate for bitmap allocator
fragmentation.
The approach is counting 'partial' and 'free' slots at L1 on the fly and
use partial / (partial+free) value as rough fragmentation estimate.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dbbe12fe6f543fc6942f41a108effc7eb1632f67)

6 years agoos/bluestore: rename new bitmap allocator class to BitmapAllocator.
Igor Fedotov [Tue, 12 Jun 2018 11:50:30 +0000 (14:50 +0300)]
os/bluestore: rename new bitmap allocator class to BitmapAllocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit a8e8c192acb48ce93553b04d1953ce8bee1b16e4)

6 years agoos/bluestore: perform allocations aligned with min_length in new bitmap allocator
Igor Fedotov [Fri, 1 Jun 2018 18:59:15 +0000 (21:59 +0300)]
os/bluestore: perform allocations aligned with min_length  in new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 51974c48e38e7080eb12c6028f95f4da666fbe76)

6 years agotest/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
Igor Fedotov [Fri, 1 Jun 2018 13:35:04 +0000 (16:35 +0300)]
test/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
ASSERT_EQ

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 69ccb4367dcdcc51fbcfdaaea308c761291621e1)

6 years agoos/bluestore: respect min_length as allocation granularity for new
Igor Fedotov [Thu, 31 May 2018 13:53:31 +0000 (16:53 +0300)]
os/bluestore: respect min_length as allocation granularity for new
bitmap allocator.
It was used a real minimum threshold before this fix which allowed e.g.
allocated extent length to be equal to min_length + 1.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 8a188b2aa3892b7f5a6c85042ffae8c2847ca869)

6 years agoos/bluestore: cosmetic new allocator internal method rename.
Igor Fedotov [Thu, 31 May 2018 11:28:28 +0000 (14:28 +0300)]
os/bluestore: cosmetic new allocator internal method rename.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fa06ff234f11f266bb10d7fa9ea28847e5ce72fe)

6 years agoos/bluestore: properly respect min_length when analysing partially free
Igor Fedotov [Thu, 31 May 2018 10:53:37 +0000 (13:53 +0300)]
os/bluestore: properly respect min_length when analysing partially free
slots in bitmap allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 288bbd42531bebd8f6c619c848e75bfd9b8f93fd)

6 years agoos/bluestore: cosmetic cleanup in new bitmap allocator.
Igor Fedotov [Thu, 31 May 2018 10:34:21 +0000 (13:34 +0300)]
os/bluestore: cosmetic cleanup in new bitmap allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 3db9d9a66cbd62c543cdd88ff4ab636fba2c2a2b)

6 years agoos/bluestore: more verbose logging in new bitmap allocator
Igor Fedotov [Wed, 30 May 2018 12:24:05 +0000 (15:24 +0300)]
os/bluestore: more verbose logging in new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 160c73032db2e122f5e643090c86eed85e679eb4)

6 years agomimic: qa/ceph-ansible: modifications for mimic branch 27014/head
Brad Hubbard [Thu, 14 Mar 2019 23:47:11 +0000 (09:47 +1000)]
mimic: qa/ceph-ansible: modifications for mimic branch

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #26860 from ashishkumsingh/wip-38648-mimic
Nathan Cutler [Fri, 15 Mar 2019 15:44:26 +0000 (16:44 +0100)]
Merge pull request #26860 from ashishkumsingh/wip-38648-mimic

mimic: doc: Fix incorrect mention of 'osd_deep_mon_scrub_interval'

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoos/bluestore: remove original bitmap allocator
Igor Fedotov [Mon, 14 May 2018 13:09:22 +0000 (16:09 +0300)]
os/bluestore: remove original bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 95bd72fc22aa313112de096ac3ef98e45c71bce5)

6 years agoos/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with...
Igor Fedotov [Fri, 11 May 2018 21:26:49 +0000 (00:26 +0300)]
os/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with min_alloc_size

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 1b6a56e80066decf8c3727090a2cd51ba9ded9bc)

6 years agoos/bluestore: fix improper access to a BitmapFastAllocator::last_pos
Igor Fedotov [Thu, 10 May 2018 15:54:36 +0000 (18:54 +0300)]
os/bluestore: fix improper access to a BitmapFastAllocator::last_pos
from multiple threads.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0c69672fce55362a7e6eb8ed6955f163afe85482)

6 years agotest/allocator: move bluestore allocator's benchmarks to a standalone UT
Igor Fedotov [Fri, 4 May 2018 17:03:37 +0000 (20:03 +0300)]
test/allocator: move bluestore allocator's benchmarks to a standalone UT

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ac2ec80cddc51f1642849c7bf226e9bed116c0d6)

6 years agoos/bluestore: add new bitmap allocator
Igor Fedotov [Thu, 3 May 2018 17:22:15 +0000 (20:22 +0300)]
os/bluestore: add new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 631041a7b788eaaf010f1e39f58b5546ab5580fc)

6 years agotest/allocator: get rid off bitmap allocator specifics and introduce new
Igor Fedotov [Thu, 3 May 2018 16:25:01 +0000 (19:25 +0300)]
test/allocator: get rid off bitmap allocator specifics and introduce new
performance test cases

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dcd30610fb1c87711c9f43f7d3d6636888792345)

6 years agoos/bluestore: add release(PExtentVector) helper to Allocator class to
Igor Fedotov [Mon, 7 May 2018 13:58:43 +0000 (16:58 +0300)]
os/bluestore: add release(PExtentVector) helper to Allocator class to
free PExtentVector explicitly.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 328ea72a84edef4a6c4e0558dff52d0e85f05122)

6 years agoos/bluestore: get rid off allocator's reserve method
Igor Fedotov [Mon, 7 May 2018 13:26:03 +0000 (16:26 +0300)]
os/bluestore: get rid off allocator's reserve method

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 88bea46242d4eb6bdfd8309ddf8843c9a14e5f25)

6 years agoMerge pull request #26968 from pdvian/wip-38668-mimic
Nathan Cutler [Fri, 15 Mar 2019 11:29:36 +0000 (12:29 +0100)]
Merge pull request #26968 from pdvian/wip-38668-mimic

mimic: doc/radosgw: Document mappings of S3 Operations to ACL grants

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc/radosgw: Document mappings of S3 Operations to ACL grants 26968/head
Adam C. Emerson [Thu, 7 Mar 2019 20:18:42 +0000 (15:18 -0500)]
doc/radosgw: Document mappings of S3 Operations to ACL grants

Since the ACL documentation is a bit sparse, let people know exactly
what operations they can expect to enable with a given ACL grant.

Fixes: https://tracker.ceph.com/issues/38523
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit bd94c93fada952537c7015743db55bff7619afa8)

6 years agoqa: ignore slow metadata io wrn during osd thrash 26961/head
Patrick Donnelly [Fri, 8 Mar 2019 18:49:10 +0000 (10:49 -0800)]
qa: ignore slow metadata io wrn during osd thrash

Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5abcc32ff642b03c4b53e565e995ebb94802579a)

6 years agorgw: bucket limit check misbehaves for > max-entries buckets (usually 1000) 26945/head
Matt Benjamin [Wed, 6 Mar 2019 22:05:11 +0000 (17:05 -0500)]
rgw: bucket limit check misbehaves for > max-entries buckets (usually 1000)

RGWBucketAdminOp::limit_check does not correctly handle
truncation/continuation markers across calls to rgw_read_user_buckets(...).
Also, it should, but does not, flush its formatter periodically.

Fixes: https://tracker.ceph.com/issues/35973
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 675b775ea52e0d7b81efb32c6cffa6987190184b)

6 years agomgr/dashboard: Search broken for entries with null values 26944/head
Patrick Nawracay [Tue, 5 Mar 2019 11:33:44 +0000 (12:33 +0100)]
mgr/dashboard: Search broken for entries with null values

Fixes: http://tracker.ceph.com/issues/38583
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
(cherry picked from commit ee988b90b84a484811b3351ff9eee242ae5b2ebd)

Conflicts:
- src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table/table.component.ts
    Modified the file to keep changes to relevant to this version.

6 years ago13.2.5 v13.2.5
Jenkins Build Slave User [Tue, 12 Mar 2019 16:48:03 +0000 (16:48 +0000)]
13.2.5

6 years agoMerge pull request #26735 from aclamk/mimic-fix-1600138
Yuri Weinstein [Fri, 8 Mar 2019 20:39:07 +0000 (12:39 -0800)]
Merge pull request #26735 from aclamk/mimic-fix-1600138

mimic: os/bluestore: Fix bluefs - block balance.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #26789 from theanalyst/mimic-resharding-fixes
Yuri Weinstein [Fri, 8 Mar 2019 20:37:43 +0000 (12:37 -0800)]
Merge pull request #26789 from theanalyst/mimic-resharding-fixes

mimic: rgw: resharding fixes

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agodoc: Fix incorrect mention of 'osd_deep_mon_scrub_interval' 26860/head
Ashish Singh [Tue, 19 Feb 2019 19:39:36 +0000 (01:09 +0530)]
doc: Fix incorrect mention of 'osd_deep_mon_scrub_interval'

Fixed the incorrect mention of 'osd_deep_mon_scrub_interval' in health-checks.rst.
Changed it to 'osd_deep_scrub_interval'.

Fixes: https://tracker.ceph.com/issues/38310
Signed-off-by: Ashish Singh <assingh@redhat.com>
(cherry picked from commit 7108e6a3c7ec1f3ae3c39682f43db99efb2b7a6c)

Conflicts:
doc/rados/operations/health-checks.rst
           - Fixed conflicts to keep relevant changes.

6 years agoMerge pull request #26814 from tspmelo/wip-fix-resolve-mimic
Abhishek L [Thu, 7 Mar 2019 13:34:32 +0000 (14:34 +0100)]
Merge pull request #26814 from tspmelo/wip-fix-resolve-mimic

mimic: mgr/dashboard: Fix dashboard compile error

Reviewed-By: Sebastian Wagner <swagner@suse.com>
Reviewed-By: Laura Paduano <lpaduano@suse.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/dashboard: Fix dashboard compile error 26814/head
Tiago Melo [Thu, 7 Mar 2019 10:48:52 +0000 (10:48 +0000)]
mgr/dashboard: Fix dashboard compile error

New version of lodash was generating a compile error.
With this change npm will install an older version of lodash.

Fixes: https://tracker.ceph.com/issues/38590
Fixes: https://tracker.ceph.com/issues/38602
Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agorgw: fix typo 26789/head
J. Eric Ivancich [Wed, 28 Nov 2018 17:55:35 +0000 (12:55 -0500)]
rgw: fix typo

This fixes a typo in a log message. It's a separate commit so
downstream commits point to the right upstream commits via
cherry-pick.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 7d1768f2a332c5ac389cbb4bbeda28752e4f17bc)

6 years agorgw: perf -- remove bucket shards asynchronously rather than synchronously
J. Eric Ivancich [Tue, 20 Nov 2018 18:32:54 +0000 (13:32 -0500)]
rgw: perf -- remove bucket shards asynchronously rather than synchronously

We can now take advantage of the new asynchronous bucket shard removal
code and where we used to remove each shard synchronously now remove
them asynchronously. This would be a huge win when we have tens of
thousands of shards.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit cb0da45c5d6e4dff6514919b3b894df0eea7c1e4)

 Conflicts:
src/rgw/rgw_rados.cc
conflicts with placement set and rgw rados refactor

6 years agorgw: clean up unused bucket index shards
J. Eric Ivancich [Tue, 20 Nov 2018 14:52:39 +0000 (09:52 -0500)]
rgw: clean up unused bucket index shards

Clean up old bucket index shards when a resharding is complete. Also,
when a resharding fails, clean up unfinished bucket index shards. Do
both clean-ups asynchronously.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit f84f70d451036ee99011b8e2c7b974f15d3a005a)

 Conflicts:
src/rgw/rgw_rados.h
merge conflict as bucket_placement functions were moved after the rgw rados
refactor

6 years agorgw: a little code clean-up by adding const
J. Eric Ivancich [Mon, 19 Nov 2018 21:10:52 +0000 (16:10 -0500)]
rgw: a little code clean-up by adding const

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 48e22fb76feffb551074f274986bc4125a02003f)

6 years agorgw: remove never-called RGWRados member function
J. Eric Ivancich [Fri, 16 Nov 2018 22:08:46 +0000 (17:08 -0500)]
rgw: remove never-called RGWRados member function

Because RGWRados::cls_rgw_init_index is never called, remove it.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 4593778c6d830a1a0d33528507f1795aee019ce4)

6 years agorgw: rename function for clarity
J. Eric Ivancich [Wed, 7 Nov 2018 20:33:52 +0000 (15:33 -0500)]
rgw: rename function for clarity

The function cls_rgw_bucket_init was renamed to
cls_rgw_bucket_init_index in order to better describe its
functionality.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 20868bd3b893ba2e17d2bfdc63d6544dbe677a9b)

6 years agorgw: batch and process bucket instances
Abhishek Lekshmanan [Wed, 17 Oct 2018 22:44:51 +0000 (00:44 +0200)]
rgw: batch and process bucket instances

Sort through and batch bucket instances so that multiple calls to reading
current bucket info and locking can be avoided. For the most trivial case when
the bucket is already deleted we exit early with all the stale instances. When
the bucket reshard is in progress we only process the stale entries with status
done, if the bucket is available for locking then we lock down and mark the
other instances as well.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit fb9c0497621fce34283094cfe260480f9122573f)

 Conflicts:
src/rgw/rgw_bucket.cc
Get rid of the following c++17isms:
- split_tenant auto return type -> trailing return type
- tuple destructuring bind for split tenant with std::tie

6 years agocli: use rgw-admin reshard stale-instances list/delete
Abhishek Lekshmanan [Mon, 15 Oct 2018 17:34:28 +0000 (19:34 +0200)]
cli: use rgw-admin reshard stale-instances list/delete

Add a delete command as well that clears the resharded instances. We print out
the json status to indicate success or error state

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit bf8f885016a588130f05c25fdef3d3b6e1216c40)

6 years agorgw: implement listing stale instances from a reshard
Abhishek Lekshmanan [Fri, 12 Oct 2018 12:55:48 +0000 (14:55 +0200)]
rgw: implement listing stale instances from a reshard

Dynamic resharding used to leave behind stale bucket instances; walk through the
metadata pool and identify these instances by comparing the reshard status. If
the reshard status is done, these instances are ok to be cleared. For reshard
status of none we compare against the bucket entry point to ensure that we don't
match the current entry point.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0c35a6fab250a817b2752f319b5c9f1da13bb3df)

6 years agoceph-volume: set number of osd ports in the tests 26702/head
Andrew Schoen [Mon, 4 Mar 2019 20:43:37 +0000 (14:43 -0600)]
ceph-volume: set number of osd ports in the tests

The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1132cb3c25fef9f853488cc8aab1327aff6bcf96)

6 years agoos/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thr... 26735/head
Adam Kupczyk [Thu, 28 Feb 2019 08:24:45 +0000 (09:24 +0100)]
os/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thread is sleeping.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
6 years agolibrbd: set *image_size to 0 when getting ENOENT in rbd_group_image_list() 26707/head
songweibin [Mon, 25 Feb 2019 06:59:25 +0000 (14:59 +0800)]
librbd: set *image_size to 0 when getting ENOENT in rbd_group_image_list()

Fixes: http://tracker.ceph.com/issues/38468
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 141e3a74b6fb10ca923c61dc90e7573f1becf002)

6 years agorbd/pybind: add __enter__ and __exit__ for class Group
songweibin [Mon, 25 Feb 2019 06:58:44 +0000 (14:58 +0800)]
rbd/pybind: add __enter__ and __exit__ for class Group

Fixes: http://tracker.ceph.com/issues/38468
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit bb8df22e4ae851d858b081a47f063a1f33b4d0c9)

6 years agomgr: drop GIL in get_config 26612/head
John Spray [Wed, 14 Nov 2018 11:19:14 +0000 (06:19 -0500)]
mgr: drop GIL in get_config

Fixes: https://tracker.ceph.com/issues/35985
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d590a5391876eda23ef286af5800e5bdbb2b4b97)

Conflicts:
        src/mgr/ActivePyModules.cc

6 years agoceph-volume: use our own testinfra suite for functional testing
Andrew Schoen [Wed, 27 Feb 2019 21:14:03 +0000 (15:14 -0600)]
ceph-volume: use our own testinfra suite for functional testing

This allows us to run a smaller subset of tests from the ones provided
by ceph-ansible and test only the things that pertain to ceph-volume.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ccfc8e6dc42b55f14fa3b4ef210cb639b9a7e20b)

6 years agorgw_file: use correct secret key to check auth 26687/head
MinSheng Lin [Fri, 25 Jan 2019 03:20:17 +0000 (03:20 +0000)]
rgw_file: use correct secret key to check auth

Currently, the authorize function in rgw_file only use the secret key of
first key pair to compare, add a function which get secret key by user's
access key and check the result is match with user's secret key.

Fixes: https://tracker.ceph.com/issues/37855
Signed-off-by: MinSheng Lin <minsheng.l@inwinstack.com>
(cherry picked from commit e92755c60325475342afcafcd5293a20e3e5216a)

6 years agoMerge pull request #26451 from smithfarm/wip-38342-mimic
Yuri Weinstein [Wed, 27 Feb 2019 16:29:26 +0000 (08:29 -0800)]
Merge pull request #26451 from smithfarm/wip-38342-mimic

mimic: core: luminous->(mimic,nautilus): PGMapDigest decode error on luminous end

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agorgw: bug in versioning concurrent, list and get have consistency issue 26664/head
wanghao72 [Wed, 30 Jan 2019 07:18:06 +0000 (15:18 +0800)]
rgw: bug in versioning concurrent, list and get have consistency issue

Fixes: https://tracker.ceph.com/issues/38060
Signed-off-by: Wang Hao <wanghao72@baidu.com>
(cherry picked from commit cc2a96a47fd71dc8399e31f1a9e87d8a13ea3419)

6 years agoMerge pull request #26590 from theanalyst/mimic-civetweb-fixes
Abhishek L [Tue, 26 Feb 2019 16:57:56 +0000 (17:57 +0100)]
Merge pull request #26590 from theanalyst/mimic-civetweb-fixes

mimic: rgw: bump mimic civetweb with ssl & abs url fixes

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Yuri Weinstein <yuriw@redhat.com>
6 years agomessages/MMonMgrDigest: use set_data() to avoid screwing up byte_throttler 26451/head
Sage Weil [Mon, 25 Feb 2019 18:41:01 +0000 (12:41 -0600)]
messages/MMonMgrDigest: use set_data() to avoid screwing up byte_throttler

Use the existing set_data() helper to avoid breaking the byte_throttler
accounting.

Fixes e4ae368ff7a5396194f8bdd5692429af5457998b

Fixes: http://tracker.ceph.com/issues/38295
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit df32d57638ecfcfe2bc88799221e0a85ad37cd38)

6 years agomessage/MMonMgrReport: conditionally reencode PGMapDigest
Sage Weil [Wed, 13 Feb 2019 13:29:45 +0000 (07:29 -0600)]
message/MMonMgrReport: conditionally reencode PGMapDigest

The encoding changed between luminous and mimic.  Reencode the
structure if the target doesn't have all the features.

Fixes: http://tracker.ceph.com/issues/38295
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e4ae368ff7a5396194f8bdd5692429af5457998b)

Conflicts:
src/CMakeLists.txt
        src/messages/MMonMgrReport.h
- mimic has msgr V1
- do not check SERVER_NAUTILUS on mimic
- use begin() instead of cbegin() on mimic

6 years agoMerge pull request #26200 from b-ranto/wip-restful-mimic
Yuri Weinstein [Mon, 25 Feb 2019 20:38:20 +0000 (12:38 -0800)]
Merge pull request #26200 from b-ranto/wip-restful-mimic

mimic: mgr/restful: fix py got exception when get osd info

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26283 from smithfarm/wip-38141-mimic
Yuri Weinstein [Mon, 25 Feb 2019 20:37:29 +0000 (12:37 -0800)]
Merge pull request #26283 from smithfarm/wip-38141-mimic

mimic: tests: Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #26426 from smithfarm/wip-38319-mimic
Yuri Weinstein [Mon, 25 Feb 2019 20:36:23 +0000 (12:36 -0800)]
Merge pull request #26426 from smithfarm/wip-38319-mimic

mimic: mgr: deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too early

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #26593 from smithfarm/wip-38451-mimic
Nathan Cutler [Fri, 22 Feb 2019 16:33:40 +0000 (17:33 +0100)]
Merge pull request #26593 from smithfarm/wip-38451-mimic

mimic: build/ops: Destruction of basic_string _GLIBCXX_USE_CXX11_ABI=0 and C++17 mode results in invalid delete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoadd a note on rgw civetweb cve in PendingReleaseNotes 26590/head
Abhishek Lekshmanan [Fri, 22 Feb 2019 16:16:12 +0000 (17:16 +0100)]
add a note on rgw civetweb cve in PendingReleaseNotes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoceph.spec: force use of upgrade devtoolset-gcc under RHEL 7 26593/head
Jason Dillaman [Tue, 5 Feb 2019 14:40:02 +0000 (09:40 -0500)]
ceph.spec: force use of upgrade devtoolset-gcc under RHEL 7

There is a known crash under C++17 mode that requires an updated
devtoolset to address.

See https://bugzilla.redhat.com/show_bug.cgi?id=1616223

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a7ce5fb07aa91bfb329dd09f036e5b650f24f2d5)

6 years agorpm: bump up required GCC version to 7.3.1
Kefu Chai [Mon, 17 Sep 2018 14:22:59 +0000 (22:22 +0800)]
rpm: bump up required GCC version to 7.3.1

so we can include the fix of
https://gcc.gnu.org/ml/libstdc++/2017-07/msg00077.html and the fix at
https://github.com/gcc-mirror/gcc/commit/7c78f2e67270187dd755632eafe3a526f4e6df30
. otherwise, if a class offers implicit conversion operator to
std::string and std::string_view, seastar will fail to compile. like

/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:3946:7:
note: candidate: std::basic_string<_CharT, _Traits, _Alloc>&
std::basic_string<_CharT, _Traits, _Alloc>::operator+=(const
std::basic_string<_CharT, _Traits, _Alloc>&) [with _CharT = char;
_Traits = std::char_traits<char>; _Alloc = std::allocator<char>]
       operator+=(const basic_string& __str)
       ^~~~~~~~
/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:3988:7:
note: candidate: std::basic_string<_CharT, _Traits, _Alloc>&
std::basic_string<_CharT, _Traits,
_Alloc>::operator+=(std::basic_string<_CharT, _Traits,
_Alloc>::__sv_type) [with _CharT = char; _Traits =
std::char_traits<char>; _Alloc = std::allocator<char>;
std::basic_string<_CharT, _Traits, _Alloc>::__sv_type =
std::basic_string_view<char>]
       operator+=(__sv_type __sv)
       ^~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5bcc179bc4ac756e0a6c81cb19fffcc508ecbcd0)

6 years agorgw: bump mimic civetweb with ssl & abs url fixes
Abhishek Lekshmanan [Fri, 22 Feb 2019 14:27:05 +0000 (15:27 +0100)]
rgw: bump mimic civetweb with ssl & abs url fixes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #26585 from capri1989/doc-fix-mimic
Nathan Cutler [Fri, 22 Feb 2019 12:30:03 +0000 (13:30 +0100)]
Merge pull request #26585 from capri1989/doc-fix-mimic

mimic: doc: Fix default value of 'mon_osd_max_split_count'

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc: Fix default value of 'mon_osd_max_split_count' 26585/head
Kai Wagner [Fri, 22 Feb 2019 10:25:06 +0000 (11:25 +0100)]
doc: Fix default value of 'mon_osd_max_split_count'

This commit is not cherry-picked from master because the config
parameter mon_osd_max_split_count has been dropped in master

Signed-off-by: Kai Wagner <kwagner@suse.com>
6 years agocommon/ceph_timer: stop timer's thread when it is suspended 26583/head
Peng Wang [Sun, 17 Feb 2019 09:47:52 +0000 (17:47 +0800)]
common/ceph_timer: stop timer's thread when it is suspended

otherwise the client could hang when shutting down

Fixes: https://tracker.ceph.com/issues/37766
Signed-off-by: Peng Wang <wp389@126.com>
(cherry picked from commit a942722c5bffbacc8d30643dd9863ef844f0617e)

6 years agotest: Verify a log trim trims the dup_index 26578/head
Brad Hubbard [Wed, 20 Feb 2019 05:47:26 +0000 (15:47 +1000)]
test: Verify a log trim trims the dup_index

Fixes: https://tracker.ceph.com/issues/38406
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e91e2f81340846e98ab0704d92a7ad543c416f1b)

6 years agoMerge pull request #26442 from xiexingguo/wip-mimic
Yuri Weinstein [Thu, 21 Feb 2019 22:21:39 +0000 (14:21 -0800)]
Merge pull request #26442 from xiexingguo/wip-mimic

mimic: Revert "msg/async: bump global_seq when retrying connection"

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26438 from pdvian/wip-38317-mimic
Yuri Weinstein [Thu, 21 Feb 2019 22:11:39 +0000 (14:11 -0800)]
Merge pull request #26438 from pdvian/wip-38317-mimic

mimic: os/filestore: ceph_abort() on fsync(2) or fdatasync(2) failure

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #26573 from neha-ojha/wip-38083-mimic
Neha Ojha [Thu, 21 Feb 2019 21:35:26 +0000 (13:35 -0800)]
Merge pull request #26573 from neha-ojha/wip-38083-mimic

mimic: qa/workunits/cephtool/test_kvstore_tool.sh: run test in ., not /tmp

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoqa/workunits/cephtool/test_kvstore_tool.sh: run test in ., not /tmp 26573/head
Sage Weil [Tue, 18 Sep 2018 12:38:59 +0000 (07:38 -0500)]
qa/workunits/cephtool/test_kvstore_tool.sh: run test in ., not /tmp

Notably, we can't make a bluestore store on a tmpfs.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5940487be3c848cf45d251f0a00a5d85a5b584e4)

6 years agoMerge pull request #26493 from dzafman/wip-alternate-backport
Yuri Weinstein [Thu, 21 Feb 2019 17:34:39 +0000 (09:34 -0800)]
Merge pull request #26493 from dzafman/wip-alternate-backport

mimic: core: scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #26285 from smithfarm/wip-38185-mimic
Yuri Weinstein [Wed, 20 Feb 2019 21:29:58 +0000 (13:29 -0800)]
Merge pull request #26285 from smithfarm/wip-38185-mimic

mimic: tests: krbd discard no longer guarantees zeroing

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
6 years agoMerge pull request #26424 from smithfarm/wip-38103-mimic
Yuri Weinstein [Wed, 20 Feb 2019 21:29:28 +0000 (13:29 -0800)]
Merge pull request #26424 from smithfarm/wip-38103-mimic

mimic: client: session flush does not cause cap release message flush

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26472 from pdvian/wip-38334-mimic
Yuri Weinstein [Wed, 20 Feb 2019 21:28:46 +0000 (13:28 -0800)]
Merge pull request #26472 from pdvian/wip-38334-mimic

mimic: cephfs: MDCache::finish_snaprealm_reconnect() create and drop MClientSnap message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #26474 from pdvian/wip-38335-mimic
Yuri Weinstein [Wed, 20 Feb 2019 21:28:09 +0000 (13:28 -0800)]
Merge pull request #26474 from pdvian/wip-38335-mimic

mimic: mds: fix potential re-evaluate stray dentry in _unlink_local_finish

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>