]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoEventEpoll: refactor del_event() a bit 27226/head
Roman Penyaev [Tue, 19 Mar 2019 10:55:52 +0000 (11:55 +0100)]
EventEpoll: refactor del_event() a bit

The main purpose of the patch is to set 'ee' members only on EPOLL_CTL_MOD
path, deletion path does not requie any of the members set.

This will make code a bit better.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit 2292120ffd2730a20fd2750a605952982f031c83)

6 years agoEventEpoll: set EPOLLET flag on del_event()
Roman Penyaev [Tue, 5 Mar 2019 12:22:59 +0000 (13:22 +0100)]
EventEpoll: set EPOLLET flag on del_event()

When new fd is being added to epoll fd EPOLLET (edge triggered) flag
is set, i.e. we assume that in and out works in edge triggered mode.

But on del_event(), when eventually epoll_ctl(EPOLL_CTL_MOD) is
invoked, EPOLLET flag is cleared, i.e. level triggered mode is
enabled.

Do not lose EPOLLET.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit c415b9fed4fbecd2fb0709fa0c0c783102f60e01)

6 years agoMerge pull request #26758 from pdvian/wip-38510-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:56:26 +0000 (09:56 -0700)]
Merge pull request #26758 from pdvian/wip-38510-luminous

luminous: CLI: ability to change file ownership

6 years agoMerge pull request #26830 from pdvian/wip-38562-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:49:48 +0000 (09:49 -0700)]
Merge pull request #26830 from pdvian/wip-38562-luminous

luminous: mgr/BaseMgrModule: drop GIL for ceph_send_command

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26900 from pdvian/wip-38586-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:48:50 +0000 (09:48 -0700)]
Merge pull request #26900 from pdvian/wip-38586-luminous

luminous: OSD crashes in get_str_map while creating with ceph-volume

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26962 from pdvian/wip-38665-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:48:21 +0000 (09:48 -0700)]
Merge pull request #26962 from pdvian/wip-38665-luminous

luminous: qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27033 from cbodley/wip-38796
Nathan Cutler [Fri, 22 Mar 2019 13:22:02 +0000 (14:22 +0100)]
Merge pull request #27033 from cbodley/wip-38796

luminous: doc/rgw: document placement target configuration

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26557 from dzafman/wip-38244
David Zafman [Tue, 19 Mar 2019 23:59:39 +0000 (16:59 -0700)]
Merge pull request #26557 from dzafman/wip-38244

luminous: scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26793 from dzafman/wip-38274
David Zafman [Tue, 19 Mar 2019 23:48:17 +0000 (16:48 -0700)]
Merge pull request #26793 from dzafman/wip-38274

luminous: Fix recovery and backfill priority handling

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26544 from ifed01/wip-ifed-fix-compress-luminous
Yuri Weinstein [Mon, 18 Mar 2019 18:59:14 +0000 (11:59 -0700)]
Merge pull request #26544 from ifed01/wip-ifed-fix-compress-luminous

luminous: os/bluestore: do not assert on non-zero err codes from compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agodoc/rgw: document placement target configuration 27033/head
Casey Bodley [Fri, 15 Mar 2019 18:35:13 +0000 (14:35 -0400)]
doc/rgw: document placement target configuration

Fixes: http://tracker.ceph.com/issues/24508
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c95585b1984e60708addee436a774d0c92ce57a6)

6 years agoMerge pull request #26597 from smithfarm/wip-release-notes-12_2_12-luminous
Nathan Cutler [Fri, 15 Mar 2019 16:18:32 +0000 (17:18 +0100)]
Merge pull request #26597 from smithfarm/wip-release-notes-12_2_12-luminous

luminous: doc: move keyring caps RN to correct point release

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #26861 from ashishkumsingh/wip-38647-luminous
Nathan Cutler [Fri, 15 Mar 2019 15:43:41 +0000 (16:43 +0100)]
Merge pull request #26861 from ashishkumsingh/wip-38647-luminous

luminous: doc: Fix incorrect mention of 'osd_deep_mon_scrub_interval'

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26372 from pdvian/wip-38095-luminous
Nathan Cutler [Fri, 15 Mar 2019 15:19:41 +0000 (16:19 +0100)]
Merge pull request #26372 from pdvian/wip-38095-luminous

luminous: doc: rados/configuration: refresh osdmap section

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26969 from pdvian/wip-38667-luminous
Nathan Cutler [Fri, 15 Mar 2019 11:31:40 +0000 (12:31 +0100)]
Merge pull request #26969 from pdvian/wip-38667-luminous

luminous: doc/radosgw: Document mappings of S3 Operations to ACL grants

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc/radosgw: Document mappings of S3 Operations to ACL grants 26969/head
Adam C. Emerson [Thu, 7 Mar 2019 20:18:42 +0000 (15:18 -0500)]
doc/radosgw: Document mappings of S3 Operations to ACL grants

Since the ACL documentation is a bit sparse, let people know exactly
what operations they can expect to enable with a given ACL grant.

Fixes: https://tracker.ceph.com/issues/38523
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit bd94c93fada952537c7015743db55bff7619afa8)

6 years agoqa: ignore slow metadata io wrn during osd thrash 26962/head
Patrick Donnelly [Fri, 8 Mar 2019 18:49:10 +0000 (10:49 -0800)]
qa: ignore slow metadata io wrn during osd thrash

Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5abcc32ff642b03c4b53e565e995ebb94802579a)

6 years agoMerge pull request #26798 from dzafman/wip-slow-luminous
Yuri Weinstein [Thu, 14 Mar 2019 21:34:23 +0000 (14:34 -0700)]
Merge pull request #26798 from dzafman/wip-slow-luminous

luminous: qa: Add "slow request" where missing based on REQUEST_SLOW

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26574 from badone/wip-luminous-test-dup_index-trim
Brad Hubbard [Wed, 13 Mar 2019 04:39:45 +0000 (14:39 +1000)]
Merge pull request #26574 from badone/wip-luminous-test-dup_index-trim

luminous: test: Verify a log trim trims the dup_index

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26613 from badone/wip-luminous-drop-GIL-in-get_config
Brad Hubbard [Tue, 12 Mar 2019 23:19:11 +0000 (09:19 +1000)]
Merge pull request #26613 from badone/wip-luminous-drop-GIL-in-get_config

luminous: mgr: drop GIL in get_config

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26427 from smithfarm/wip-38318-luminous
Yuri Weinstein [Tue, 12 Mar 2019 19:53:34 +0000 (12:53 -0700)]
Merge pull request #26427 from smithfarm/wip-38318-luminous

luminous: mgr: deadlock: _check_auth_rotating possible clock skew, rotating keys expired way too early

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Tim Serong <tserong@suse.com>
6 years agoMerge pull request #26706 from pdvian/wip-38506-luminous
Yuri Weinstein [Tue, 12 Mar 2019 19:52:22 +0000 (12:52 -0700)]
Merge pull request #26706 from pdvian/wip-38506-luminous

luminous: osd/PrimaryLogPG: handle object !exists in handle_watch_timeout

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #26871 from neha-ojha/wip-38316-luminous
Yuri Weinstein [Tue, 12 Mar 2019 19:51:41 +0000 (12:51 -0700)]
Merge pull request #26871 from neha-ojha/wip-38316-luminous

luminous: core: os/filestore: ceph_abort() on fsync(2) or fdatasync(2) failure

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26677 from smithfarm/wip-38501-luminous
Yuri Weinstein [Tue, 12 Mar 2019 16:22:02 +0000 (09:22 -0700)]
Merge pull request #26677 from smithfarm/wip-38501-luminous

luminous: rgw: rgw_file: only first subuser can be exported to nfs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26791 from cbodley/wip-38608
Yuri Weinstein [Tue, 12 Mar 2019 16:19:48 +0000 (09:19 -0700)]
Merge pull request #26791 from cbodley/wip-38608

luminous: qa/rgw: reduce number of multisite log shards

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agocommon/str_map: better trim() impl using boost 26900/head
Sage Weil [Fri, 1 Mar 2019 05:41:59 +0000 (23:41 -0600)]
common/str_map: better trim() impl using boost

Thanks Kefu!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6154874aa9ae6dd66a2a0decb289f2a718f65048)

Conflicts:
src/common/str_map.cc : Resolved in trim

6 years agocommon/str_map: fix trim() on empty string
Sage Weil [Thu, 28 Feb 2019 16:38:54 +0000 (10:38 -0600)]
common/str_map: fix trim() on empty string

If was pass an empty string (e.g., with something like
get_str_map("M= T= P=")) we end up with a (size_t)-1 for end.

Fixes: http://tracker.ceph.com/issues/38329
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 16ffca6f0a23a96284a7d17c890047121d32058e)

6 years agoMerge pull request #25810 from pdvian/wip-37741-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:45:27 +0000 (12:45 -0700)]
Merge pull request #25810 from pdvian/wip-37741-luminous

luminous: build/ops: rpm: require ceph-base instead of ceph-common

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26127 from xiexingguo/wip-luminous-upmap-fixes
Yuri Weinstein [Mon, 11 Mar 2019 19:44:26 +0000 (12:44 -0700)]
Merge pull request #26127 from xiexingguo/wip-luminous-upmap-fixes

luminous: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26199 from b-ranto/wip-restful-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:42:52 +0000 (12:42 -0700)]
Merge pull request #26199 from b-ranto/wip-restful-luminous

luminous: mgr/restful: fix py got exception when get osd info

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26355 from pdvian/wip-38240-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:42:21 +0000 (12:42 -0700)]
Merge pull request #26355 from pdvian/wip-38240-luminous

luminous: qa/tasks/radosbench: default to 64k writes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26540 from ifed01/wip-ifed-38395
Yuri Weinstein [Mon, 11 Mar 2019 19:41:34 +0000 (12:41 -0700)]
Merge pull request #26540 from ifed01/wip-ifed-38395

luminous: os/bluestore: fix lack of onode ref during removal

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26579 from pdvian/wip-38400-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:40:53 +0000 (12:40 -0700)]
Merge pull request #26579 from pdvian/wip-38400-luminous

luminous: common: ceph_timer: stop timer's thread when it is suspended

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26866 from neha-ojha/wip-38574-luminous
Neha Ojha [Mon, 11 Mar 2019 16:55:29 +0000 (09:55 -0700)]
Merge pull request #26866 from neha-ojha/wip-38574-luminous

luminous: os/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thread is sleeping

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
6 years agoos/filestore: ceph_abort() on fsync(2) or fdatasync(2) failure 26871/head
Sage Weil [Mon, 11 Feb 2019 17:11:22 +0000 (11:11 -0600)]
os/filestore: ceph_abort() on fsync(2) or fdatasync(2) failure

If we get an error from f[data]sync that is always a fatal error.

Fixes: http://tracker.ceph.com/issues/38258
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3892b81303bcc35056ca06371651e13b7a4c4f2a)

 Conflicts:
src/os/filestore/FileStore.cc - encode trivial resolution
src/os/filestore/WBThrottle.cc - trivial resolution

6 years agoos/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thr... 26866/head
Adam Kupczyk [Thu, 28 Feb 2019 08:24:45 +0000 (09:24 +0100)]
os/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thread is sleeping.

Fixes: https://tracker.ceph.com/issues/38574
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit cbaa17c1fca858ce25c1e3de193151a8b7b10ef7)

Conflicts:
src/os/bluestore/BlueStore.cc: trivial resolution

6 years agodoc: Fix incorrect mention of 'osd_deep_mon_scrub_interval' 26861/head
Ashish Singh [Tue, 19 Feb 2019 19:39:36 +0000 (01:09 +0530)]
doc: Fix incorrect mention of 'osd_deep_mon_scrub_interval'

Fixed the incorrect mention of 'osd_deep_mon_scrub_interval' in health-checks.rst.
Changed it to 'osd_deep_scrub_interval'.

Fixes: https://tracker.ceph.com/issues/38310
Signed-off-by: Ashish Singh <assingh@redhat.com>
(cherry picked from commit 7108e6a3c7ec1f3ae3c39682f43db99efb2b7a6c)

Conflicts:
doc/rados/operations/health-checks.rst
    - Fixed conflicts to keep relevant changes.

6 years agoMerge pull request #26857 from ceph/backport-luminous-26826
Alfredo Deza [Fri, 8 Mar 2019 18:49:02 +0000 (13:49 -0500)]
Merge pull request #26857 from ceph/backport-luminous-26826

luminous: ceph-volume: `simple scan` will now scan all running ceph-disk OSDs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume: make systemctl.get_running_osds resilient to garbage output 26857/head
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f3707f1ad43513c2a6df17e9dce756533c3b4a1d)

6 years agoceph-volume: remove obsoleted simple unit test
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b59dcd01c2dd386ca1b63eb66ed243afffb80211)

6 years agoceph-volume: tox tests should ignore the tests/functional directory
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1881f386d8207ee8c10f0863c4a1abf18cf3c5d9)

6 years agoceph-volume: use `simple scan` and `simple activate --all` in tests
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 87c33f149a83bb1175bfb473b226e8c46e4e2a6e)

6 years agoceph-volume: update docs and help menu for `simple scan`
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f7943ef6007fb05d83602bc22cbacf3bb67d8625)

6 years agoceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7cb098b6899542c875d911d28771a4c450d0bb3a)

6 years agoceph-volume: update `simple scan` to scan all running OSDs
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 28e454ec30f867ae1229c66d263bcd608654553f)

6 years agoceph-volume: add systemd.systemctl.get_running_osd_ids
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 15d20e29a2f2d74caf05dfec559e30941c7a46ab)

6 years agomgr/BaseMgrModule: drop GIL for ceph_send_command 26830/head
Sage Weil [Fri, 1 Mar 2019 17:00:55 +0000 (11:00 -0600)]
mgr/BaseMgrModule: drop GIL for ceph_send_command

Otherwise, we can easily deadlock, since other bits of the code hold the
objecter lock and then take the GIL.

Fixes: http://tracker.ceph.com/issues/38537
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 94486a9fb6827157bee2d2483f20222126083b22)

Conflicts:
src/mgr/BaseMgrModule.cc : Resolved in ceph_send_command

6 years agoMerge pull request #26824 from ceph/backport-luminous-26739
Alfredo Deza [Thu, 7 Mar 2019 20:22:59 +0000 (15:22 -0500)]
Merge pull request #26824 from ceph/backport-luminous-26739

luminous: ceph-volume: replace testinfra command with py.test

Reviewed-by Alfredo Deza <adeza@redhat.com>

6 years agoceph-volume tests.simple replace testinfra command for py.test 26824/head
Alfredo Deza [Mon, 4 Mar 2019 14:05:07 +0000 (09:05 -0500)]
ceph-volume tests.simple replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e90ebc3264004cab3286895c843db43f20a4840b)

6 years agoceph-volume tests.lvm replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:51 +0000 (09:04 -0500)]
ceph-volume tests.lvm replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c4597fc63195c654b960f747f9d50a558fa719c0)

6 years agoceph-volume tests.batch replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:36 +0000 (09:04 -0500)]
ceph-volume tests.batch replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dc7501af69ccbdabb5e6a77a91696454855201e9)

6 years agoqa/rgw: reduce number of multisite log shards 26791/head
Casey Bodley [Wed, 5 Sep 2018 18:45:04 +0000 (14:45 -0400)]
qa/rgw: reduce number of multisite log shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2629231ca1782b6a02e8279182c707d56d95290a)

Conflicts:
qa/suites/rgw/multisite/overrides.yaml curl timeout

6 years agoMerge pull request #26232 from batrick/i38132-i38130
Yuri Weinstein [Wed, 6 Mar 2019 15:52:39 +0000 (07:52 -0800)]
Merge pull request #26232 from batrick/i38132-i38130

luminous: mds: stopping MDS with a large cache (40+GB) causes it to miss heartbeats

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26661 from batrick/i38488
Yuri Weinstein [Wed, 6 Mar 2019 15:51:38 +0000 (07:51 -0800)]
Merge pull request #26661 from batrick/i38488

luminous: mds: message invalid access

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa: Add "slow request" for luminous only where missing based on REQUEST_SLOW 26798/head
David Zafman [Wed, 6 Mar 2019 11:58:43 +0000 (11:58 +0000)]
qa: Add "slow request" for luminous only where missing based on REQUEST_SLOW

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoosd: Add OSD_BACKFILL_PRIORITY_FORCED with lower priority 26793/head
David Zafman [Fri, 1 Feb 2019 07:04:03 +0000 (23:04 -0800)]
osd: Add OSD_BACKFILL_PRIORITY_FORCED with lower priority
than OSD_RECOVERY_PRIORITY_FORCED

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d088ffb09f46c6207cfedd2cdab9f074197af4bb)

6 years agotest: Add scripts to test backfill/recovery priority handling
David Zafman [Sat, 26 Jan 2019 19:16:17 +0000 (11:16 -0800)]
test: Add scripts to test backfill/recovery priority handling

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ca5cf14fa83ee89260688d467257bb3d81efdd36)

6 years agoosd AsyncReserver: Update priority in AsyncReserver for force and cancel-force
David Zafman [Sat, 26 Jan 2019 00:04:11 +0000 (16:04 -0800)]
osd AsyncReserver: Update priority in AsyncReserver for force and cancel-force

Fixes: http://tracker.ceph.com/issues/38041
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 11097f1235265a3eb753adc7e6d2818e5902e39e)

Conflicts:
src/osd/PG.cc (trivial)
        src/common/AsyncReserver.h (update_priority() assert, lock for luminous)

6 years agotest: Add option to wait_for_clean() to execute at every sleep
David Zafman [Sat, 26 Jan 2019 00:03:30 +0000 (16:03 -0800)]
test: Add option to wait_for_clean() to execute at every sleep

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 70b51362084d2ae5f2a6e9e5856f17452c53e3f3)

6 years agoosd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags
xie xingguo [Tue, 4 Sep 2018 07:34:24 +0000 (15:34 +0800)]
osd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags

__set_force_backfill__ should set PG_STATE_FORCED_BACKFILL
instead of PG_STATE_FORCED_RECOVERY.

Fixes: http://tracker.ceph.com/issues/27985
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 60ef742a84d5201b2a6e6c1780741da0b31c49a8)

Conflicts:
src/osd/PG.cc (trivial)

6 years agoosd: print summary for forced backfill/recovery to debug
Sage Weil [Mon, 18 Sep 2017 21:53:31 +0000 (16:53 -0500)]
osd: print summary for forced backfill/recovery to debug

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7816ab5a8eb0ea754a0809eb9da915a16ad39e83)

Conflicts:
src/osd/PG.h (pg_id public like master for OSDService::adjust_pg_priorities())

6 years agoosd/PG: clean up set_force_{recovery,backfill} interface
Sage Weil [Wed, 13 Sep 2017 22:09:35 +0000 (18:09 -0400)]
osd/PG: clean up set_force_{recovery,backfill} interface

- update state under pg lock
- clean up PG interface
- log only when we adjust the state

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2482b4cc91579b4ef5748097f1adacaad91510b1)

Conflicts:
src/osd/PG.h (trivial)

6 years agoMerge pull request #26703 from ceph/backport-luminous-26685
Alfredo Deza [Tue, 5 Mar 2019 15:29:46 +0000 (10:29 -0500)]
Merge pull request #26703 from ceph/backport-luminous-26685

luminous: ceph-volume: use our own testinfra suite for functional testing

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoCLI: ability to change file ownership 26758/head
Sébastien Han [Mon, 18 Feb 2019 18:17:23 +0000 (19:17 +0100)]
CLI: ability to change file ownership

When creating/fetching key it's nice to have the ability to change the
ownership of the created file.

This commit adds the '--setuser' and 'setgroup' which respectively apply
the desired owner and group to a file user when '--output' is passed.

Closes: https://tracker.ceph.com/issues/38370
Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit 0e26090960213ef911cfc7bfcdf2925ddf1b268f)

Conflicts:
src/ceph.in : Resolved in  new_style_command

6 years agoMerge pull request #26686 from dzafman/wip-test-fix
Neha Ojha [Mon, 4 Mar 2019 22:28:22 +0000 (14:28 -0800)]
Merge pull request #26686 from dzafman/wip-test-fix

luminous: qa/workunits/cephtool/test.sh: Use :- to handle TMPDIR=""

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweinstei@redhat.com>
6 years agoceph-volume: set number of osd ports in the tests 26703/head
Andrew Schoen [Mon, 4 Mar 2019 20:43:37 +0000 (14:43 -0600)]
ceph-volume: set number of osd ports in the tests

The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1132cb3c25fef9f853488cc8aab1327aff6bcf96)

6 years agomds: handle negative decay counter 26232/head
Patrick Donnelly [Sat, 2 Feb 2019 00:00:13 +0000 (16:00 -0800)]
mds: handle negative decay counter

Problem only exists in Luminous/Mimic.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agotest/mds: fix Session cons call
Patrick Donnelly [Fri, 1 Feb 2019 18:07:58 +0000 (10:07 -0800)]
test/mds: fix Session cons call

Problem did not exist in master.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: simplify recall warnings
Patrick Donnelly [Mon, 28 Jan 2019 23:48:38 +0000 (15:48 -0800)]
mds: simplify recall warnings

Instead of a timeout and complicated decisions about whether the client is
releasing caps in an expeditious fashion, just use a DecayCounter that tracks
the number of caps we've recalled. This counter is decremented whenever the
client releases caps. If the counter passes a threshold, then we raise the
warning.

Similar reworking is done for the steady-state recall of client caps. Another
release DecayCounter is added so we can tell when the client is not releasing
any more caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0b3a1148475ea7e563b1b8c13217b6e7c85b150)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/Beacon.cc
src/mds/Server.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: add extra details for cache drop output
Patrick Donnelly [Fri, 25 Jan 2019 23:59:13 +0000 (15:59 -0800)]
mds: add extra details for cache drop output

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3bc093fd9e9daf99f09abe68ce0c0077ef0f58da)

6 years agoqa: test mds_max_caps_per_client conf
Patrick Donnelly [Fri, 25 Jan 2019 20:13:50 +0000 (12:13 -0800)]
qa: test mds_max_caps_per_client conf

That the MDS will not let a client sit above mds_max_caps_per_client caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 30aaa884bf3cf4ba8e344390c974906624c2258a)

6 years agomds: limit maximum number of caps held by session
Patrick Donnelly [Thu, 24 Jan 2019 22:23:08 +0000 (14:23 -0800)]
mds: limit maximum number of caps held by session

This is to prevent unsustainable situations where a client has so many
outstanding caps that a linear traversal/operation on the session's caps takes
unacceptable amounts of time.

Fixes: http://tracker.ceph.com/issues/38022
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 48ca097a9f537545b4b0c864ebb8c5e81d821098)

Conflicts:
PendingReleaseNotes
src/mds/Server.cc

6 years agomds: adapt drop cache for incremental recall
Patrick Donnelly [Thu, 24 Jan 2019 22:22:42 +0000 (14:22 -0800)]
mds: adapt drop cache for incremental recall

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7244cae4b8b4455e6b66fcaa5299905c839bee24)

6 years agomds: recall caps incrementally
Patrick Donnelly [Wed, 23 Jan 2019 14:41:55 +0000 (06:41 -0800)]
mds: recall caps incrementally

As with trimming, use DecayCounters to throttle the number of caps we recall,
both globally and per-session.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ef46216d8d0b659549925481b4eff6bd7d2c43c9)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/Beacon.cc
src/mds/Server.cc
src/mds/Server.h
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: adapt drop cache for incremental trim
Patrick Donnelly [Sun, 20 Jan 2019 04:40:11 +0000 (20:40 -0800)]
mds: adapt drop cache for incremental trim

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b750b3bb25d3c0cbad7a24ad6c7fd215d1b16395)

6 years agomds: add throttle for trimming MDCache
Patrick Donnelly [Sat, 19 Jan 2019 00:18:59 +0000 (16:18 -0800)]
mds: add throttle for trimming MDCache

This is necessary when the MDS cache size decreases by a significant amount.
For example, when stopping a large MDS or when the operator makes a large cache
size reduction.

Fixes: http://tracker.ceph.com/issues/37723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7bf2f31abf069b9bafc52bc1478292bb8a6c71b8)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/MDCache.cc
src/mds/MDCache.h

6 years agomds: cleanup SessionMap init
Patrick Donnelly [Fri, 18 Jan 2019 23:43:48 +0000 (15:43 -0800)]
mds: cleanup SessionMap init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 69efdaf6465c368795c0e564898a39baf34d09c2)

6 years agomds: cleanup Session init
Patrick Donnelly [Mon, 21 Jan 2019 18:57:45 +0000 (10:57 -0800)]
mds: cleanup Session init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ce153b811263932b42b04a7b8abe63e8241d0ddc)

Conflicts:
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: get ref to message for lambda 26661/head
Patrick Donnelly [Tue, 26 Feb 2019 20:27:40 +0000 (12:27 -0800)]
mds: get ref to message for lambda

Otherwise we have invalid reads. Caught by valgrind.

This cannot be cherry-picked as it was a bug introduced by a backport.

Introduced-by: 2f6289ff7f7385e462d59e096634a1dee0b60837
Fixes: http://tracker.ceph.com/issues/38488
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26546 from cbodley/wip-38410-luminous
Yuri Weinstein [Mon, 4 Mar 2019 17:06:17 +0000 (09:06 -0800)]
Merge pull request #26546 from cbodley/wip-38410-luminous

luminous: rgw: fix cls_bucket_head result order consistency

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26548 from cbodley/wip-38414-luminous
Yuri Weinstein [Mon, 4 Mar 2019 17:05:50 +0000 (09:05 -0800)]
Merge pull request #26548 from cbodley/wip-38414-luminous

luminous: rgw: bug in versioning concurrent, list and get have consistency issue

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoosd/PrimaryLogPG: handle object !exists in handle_watch_timeout 26706/head
Sage Weil [Fri, 22 Feb 2019 14:56:16 +0000 (08:56 -0600)]
osd/PrimaryLogPG: handle object !exists in handle_watch_timeout

- watch on object
- watch timeout queued
- rados op deletes object
- handle_watch_timeout tries to delete it again

Fixes: http://tracker.ceph.com/issues/38432
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit bdf2d1b97308835bec038b10663bc438da013e9d)

6 years agomgr: drop GIL in get_config 26613/head
John Spray [Wed, 14 Nov 2018 11:19:14 +0000 (06:19 -0500)]
mgr: drop GIL in get_config

Fixes: https://tracker.ceph.com/issues/35985
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d590a53)

Conflicts:
        src/mgr/ActivePyModules.cc

6 years agoMerge pull request #25825 from pdvian/wip-36434-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:35:25 +0000 (11:35 -0800)]
Merge pull request #25825 from pdvian/wip-36434-luminous

luminous: mon: monstore tool rebuild does not generate creating_pgs

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25855 from ashishkumsingh/wip-37825-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:30:16 +0000 (11:30 -0800)]
Merge pull request #25855 from ashishkumsingh/wip-37825-luminous

luminous: os/bluestore: KernelDevice::read() does the EIO mapping now.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #25956 from xiexingguo/wip-liminous-async-fixes
Yuri Weinstein [Thu, 28 Feb 2019 19:29:28 +0000 (11:29 -0800)]
Merge pull request #25956 from xiexingguo/wip-liminous-async-fixes

luminous: msg/async: backport recent messenger fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26042 from trociny/wip-37972-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:24:46 +0000 (11:24 -0800)]
Merge pull request #26042 from trociny/wip-37972-luminous

luminous: msg/msg_types: fix the dencoder of entity_addr_t

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26284 from smithfarm/wip-38140-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:24:17 +0000 (11:24 -0800)]
Merge pull request #26284 from smithfarm/wip-38140-luminous

luminous: tests: Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #26501 from k0ste/luminous_backports2
Yuri Weinstein [Thu, 28 Feb 2019 19:23:50 +0000 (11:23 -0800)]
Merge pull request #26501 from k0ste/luminous_backports2

luminous: mgr/balancer: restrict automatic balancing to specific weekdays

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoceph-volume: use our own testinfra suite for functional testing
Andrew Schoen [Wed, 27 Feb 2019 21:14:03 +0000 (15:14 -0600)]
ceph-volume: use our own testinfra suite for functional testing

This allows us to run a smaller subset of tests from the ones provided
by ceph-ansible and test only the things that pertain to ceph-volume.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ccfc8e6dc42b55f14fa3b4ef210cb639b9a7e20b)

6 years agoMerge pull request #26656 from jan--f/c-v-simple-activate-all-luminous
Andrew Schoen [Wed, 27 Feb 2019 21:58:51 +0000 (15:58 -0600)]
Merge pull request #26656 from jan--f/c-v-simple-activate-all-luminous

luminous: ceph-volume: add --all flag to simple activate

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorgw_file: use correct secret key to check auth 26677/head
MinSheng Lin [Fri, 25 Jan 2019 03:20:17 +0000 (03:20 +0000)]
rgw_file: use correct secret key to check auth

Currently, the authorize function in rgw_file only use the secret key of
first key pair to compare, add a function which get secret key by user's
access key and check the result is match with user's secret key.

Fixes: https://tracker.ceph.com/issues/37855
Signed-off-by: MinSheng Lin <minsheng.l@inwinstack.com>
(cherry picked from commit e92755c60325475342afcafcd5293a20e3e5216a)

6 years agoMerge pull request #26065 from pdvian/wip-37989-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:18:17 +0000 (14:18 -0800)]
Merge pull request #26065 from pdvian/wip-37989-luminous

luminous: MDSMonitor: missing osdmon writeable check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25858 from ashishkumsingh/wip-37823-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:14:18 +0000 (14:14 -0800)]
Merge pull request #25858 from ashishkumsingh/wip-37823-luminous

luminous: mds: log evicted clients to clog/dbg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25979 from pdvian/wip-37908-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:13:47 +0000 (14:13 -0800)]
Merge pull request #25979 from pdvian/wip-37908-luminous

luminous: mds: wait shorter intervals if beacon not sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26215 from pdvian/wip-38102-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:11:50 +0000 (14:11 -0800)]
Merge pull request #26215 from pdvian/wip-38102-luminous

luminous: mds: trim cache after journal flush

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26293 from pdvian/wip-38190-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:11:21 +0000 (14:11 -0800)]
Merge pull request #26293 from pdvian/wip-38190-luminous

luminous: mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26473 from pdvian/wip-38336-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:10:51 +0000 (14:10 -0800)]
Merge pull request #26473 from pdvian/wip-38336-luminous

luminous: mds: fix potential re-evaluate stray dentry in _unlink_local_finish

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoceph-volume: add --all flag to simple activate 26656/head
Jan Fajerski [Thu, 31 Jan 2019 12:45:46 +0000 (13:45 +0100)]
ceph-volume: add --all flag to simple activate

This is intended to behave similarly to the lvm activate --all argument.
When passed, c-v will scan /etc/ceph/osd/ (or the location specified by
CEPH_VOLUME_SIMPLE_JSON_DIR) for json files (glob *.json) and call activate
for each file.  This should greatly ease the take-over of ceph-disk OSDs
with manual commands and deployment tools like DeepSea and ceph-ansible.
Also adds a simple unit test and modifies the
simple/centos7/filestore/activate functional test to use --all.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05af94c6fca66083107e96e9aa1db87619e77e66)

 Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
        resolved by deleting everything after conflict marker

6 years agoMerge pull request #25611 from pdvian/wip-37561-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:47:05 +0000 (12:47 -0800)]
Merge pull request #25611 from pdvian/wip-37561-luminous

luminous: rgw: rgwgc: process coredump in some special case

Reviewed-by: Casey Bodley <cbodley@redhat.com>