]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #25810 from pdvian/wip-37741-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:45:27 +0000 (12:45 -0700)]
Merge pull request #25810 from pdvian/wip-37741-luminous

luminous: build/ops: rpm: require ceph-base instead of ceph-common

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26127 from xiexingguo/wip-luminous-upmap-fixes
Yuri Weinstein [Mon, 11 Mar 2019 19:44:26 +0000 (12:44 -0700)]
Merge pull request #26127 from xiexingguo/wip-luminous-upmap-fixes

luminous: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26199 from b-ranto/wip-restful-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:42:52 +0000 (12:42 -0700)]
Merge pull request #26199 from b-ranto/wip-restful-luminous

luminous: mgr/restful: fix py got exception when get osd info

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26355 from pdvian/wip-38240-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:42:21 +0000 (12:42 -0700)]
Merge pull request #26355 from pdvian/wip-38240-luminous

luminous: qa/tasks/radosbench: default to 64k writes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26540 from ifed01/wip-ifed-38395
Yuri Weinstein [Mon, 11 Mar 2019 19:41:34 +0000 (12:41 -0700)]
Merge pull request #26540 from ifed01/wip-ifed-38395

luminous: os/bluestore: fix lack of onode ref during removal

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26579 from pdvian/wip-38400-luminous
Yuri Weinstein [Mon, 11 Mar 2019 19:40:53 +0000 (12:40 -0700)]
Merge pull request #26579 from pdvian/wip-38400-luminous

luminous: common: ceph_timer: stop timer's thread when it is suspended

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26866 from neha-ojha/wip-38574-luminous
Neha Ojha [Mon, 11 Mar 2019 16:55:29 +0000 (09:55 -0700)]
Merge pull request #26866 from neha-ojha/wip-38574-luminous

luminous: os/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thread is sleeping

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
6 years agoos/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thr... 26866/head
Adam Kupczyk [Thu, 28 Feb 2019 08:24:45 +0000 (09:24 +0100)]
os/bluestore: Fix problem with bluefs's freespace not being balanced when kv_sync_thread is sleeping.

Fixes: https://tracker.ceph.com/issues/38574
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit cbaa17c1fca858ce25c1e3de193151a8b7b10ef7)

Conflicts:
src/os/bluestore/BlueStore.cc: trivial resolution

6 years agoMerge pull request #26857 from ceph/backport-luminous-26826
Alfredo Deza [Fri, 8 Mar 2019 18:49:02 +0000 (13:49 -0500)]
Merge pull request #26857 from ceph/backport-luminous-26826

luminous: ceph-volume: `simple scan` will now scan all running ceph-disk OSDs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume: make systemctl.get_running_osds resilient to garbage output 26857/head
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f3707f1ad43513c2a6df17e9dce756533c3b4a1d)

6 years agoceph-volume: remove obsoleted simple unit test
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b59dcd01c2dd386ca1b63eb66ed243afffb80211)

6 years agoceph-volume: tox tests should ignore the tests/functional directory
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1881f386d8207ee8c10f0863c4a1abf18cf3c5d9)

6 years agoceph-volume: use `simple scan` and `simple activate --all` in tests
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 87c33f149a83bb1175bfb473b226e8c46e4e2a6e)

6 years agoceph-volume: update docs and help menu for `simple scan`
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f7943ef6007fb05d83602bc22cbacf3bb67d8625)

6 years agoceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7cb098b6899542c875d911d28771a4c450d0bb3a)

6 years agoceph-volume: update `simple scan` to scan all running OSDs
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 28e454ec30f867ae1229c66d263bcd608654553f)

6 years agoceph-volume: add systemd.systemctl.get_running_osd_ids
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 15d20e29a2f2d74caf05dfec559e30941c7a46ab)

6 years agoMerge pull request #26824 from ceph/backport-luminous-26739
Alfredo Deza [Thu, 7 Mar 2019 20:22:59 +0000 (15:22 -0500)]
Merge pull request #26824 from ceph/backport-luminous-26739

luminous: ceph-volume: replace testinfra command with py.test

Reviewed-by Alfredo Deza <adeza@redhat.com>

6 years agoceph-volume tests.simple replace testinfra command for py.test 26824/head
Alfredo Deza [Mon, 4 Mar 2019 14:05:07 +0000 (09:05 -0500)]
ceph-volume tests.simple replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e90ebc3264004cab3286895c843db43f20a4840b)

6 years agoceph-volume tests.lvm replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:51 +0000 (09:04 -0500)]
ceph-volume tests.lvm replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c4597fc63195c654b960f747f9d50a558fa719c0)

6 years agoceph-volume tests.batch replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:36 +0000 (09:04 -0500)]
ceph-volume tests.batch replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dc7501af69ccbdabb5e6a77a91696454855201e9)

6 years agoMerge pull request #26232 from batrick/i38132-i38130
Yuri Weinstein [Wed, 6 Mar 2019 15:52:39 +0000 (07:52 -0800)]
Merge pull request #26232 from batrick/i38132-i38130

luminous: mds: stopping MDS with a large cache (40+GB) causes it to miss heartbeats

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26661 from batrick/i38488
Yuri Weinstein [Wed, 6 Mar 2019 15:51:38 +0000 (07:51 -0800)]
Merge pull request #26661 from batrick/i38488

luminous: mds: message invalid access

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26703 from ceph/backport-luminous-26685
Alfredo Deza [Tue, 5 Mar 2019 15:29:46 +0000 (10:29 -0500)]
Merge pull request #26703 from ceph/backport-luminous-26685

luminous: ceph-volume: use our own testinfra suite for functional testing

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #26686 from dzafman/wip-test-fix
Neha Ojha [Mon, 4 Mar 2019 22:28:22 +0000 (14:28 -0800)]
Merge pull request #26686 from dzafman/wip-test-fix

luminous: qa/workunits/cephtool/test.sh: Use :- to handle TMPDIR=""

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweinstei@redhat.com>
6 years agoceph-volume: set number of osd ports in the tests 26703/head
Andrew Schoen [Mon, 4 Mar 2019 20:43:37 +0000 (14:43 -0600)]
ceph-volume: set number of osd ports in the tests

The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1132cb3c25fef9f853488cc8aab1327aff6bcf96)

6 years agomds: handle negative decay counter 26232/head
Patrick Donnelly [Sat, 2 Feb 2019 00:00:13 +0000 (16:00 -0800)]
mds: handle negative decay counter

Problem only exists in Luminous/Mimic.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agotest/mds: fix Session cons call
Patrick Donnelly [Fri, 1 Feb 2019 18:07:58 +0000 (10:07 -0800)]
test/mds: fix Session cons call

Problem did not exist in master.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: simplify recall warnings
Patrick Donnelly [Mon, 28 Jan 2019 23:48:38 +0000 (15:48 -0800)]
mds: simplify recall warnings

Instead of a timeout and complicated decisions about whether the client is
releasing caps in an expeditious fashion, just use a DecayCounter that tracks
the number of caps we've recalled. This counter is decremented whenever the
client releases caps. If the counter passes a threshold, then we raise the
warning.

Similar reworking is done for the steady-state recall of client caps. Another
release DecayCounter is added so we can tell when the client is not releasing
any more caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0b3a1148475ea7e563b1b8c13217b6e7c85b150)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/Beacon.cc
src/mds/Server.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: add extra details for cache drop output
Patrick Donnelly [Fri, 25 Jan 2019 23:59:13 +0000 (15:59 -0800)]
mds: add extra details for cache drop output

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3bc093fd9e9daf99f09abe68ce0c0077ef0f58da)

6 years agoqa: test mds_max_caps_per_client conf
Patrick Donnelly [Fri, 25 Jan 2019 20:13:50 +0000 (12:13 -0800)]
qa: test mds_max_caps_per_client conf

That the MDS will not let a client sit above mds_max_caps_per_client caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 30aaa884bf3cf4ba8e344390c974906624c2258a)

6 years agomds: limit maximum number of caps held by session
Patrick Donnelly [Thu, 24 Jan 2019 22:23:08 +0000 (14:23 -0800)]
mds: limit maximum number of caps held by session

This is to prevent unsustainable situations where a client has so many
outstanding caps that a linear traversal/operation on the session's caps takes
unacceptable amounts of time.

Fixes: http://tracker.ceph.com/issues/38022
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 48ca097a9f537545b4b0c864ebb8c5e81d821098)

Conflicts:
PendingReleaseNotes
src/mds/Server.cc

6 years agomds: adapt drop cache for incremental recall
Patrick Donnelly [Thu, 24 Jan 2019 22:22:42 +0000 (14:22 -0800)]
mds: adapt drop cache for incremental recall

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7244cae4b8b4455e6b66fcaa5299905c839bee24)

6 years agomds: recall caps incrementally
Patrick Donnelly [Wed, 23 Jan 2019 14:41:55 +0000 (06:41 -0800)]
mds: recall caps incrementally

As with trimming, use DecayCounters to throttle the number of caps we recall,
both globally and per-session.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ef46216d8d0b659549925481b4eff6bd7d2c43c9)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/Beacon.cc
src/mds/Server.cc
src/mds/Server.h
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: adapt drop cache for incremental trim
Patrick Donnelly [Sun, 20 Jan 2019 04:40:11 +0000 (20:40 -0800)]
mds: adapt drop cache for incremental trim

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b750b3bb25d3c0cbad7a24ad6c7fd215d1b16395)

6 years agomds: add throttle for trimming MDCache
Patrick Donnelly [Sat, 19 Jan 2019 00:18:59 +0000 (16:18 -0800)]
mds: add throttle for trimming MDCache

This is necessary when the MDS cache size decreases by a significant amount.
For example, when stopping a large MDS or when the operator makes a large cache
size reduction.

Fixes: http://tracker.ceph.com/issues/37723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7bf2f31abf069b9bafc52bc1478292bb8a6c71b8)

Conflicts:
PendingReleaseNotes
src/common/options.cc
src/mds/MDCache.cc
src/mds/MDCache.h

6 years agomds: cleanup SessionMap init
Patrick Donnelly [Fri, 18 Jan 2019 23:43:48 +0000 (15:43 -0800)]
mds: cleanup SessionMap init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 69efdaf6465c368795c0e564898a39baf34d09c2)

6 years agomds: cleanup Session init
Patrick Donnelly [Mon, 21 Jan 2019 18:57:45 +0000 (10:57 -0800)]
mds: cleanup Session init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ce153b811263932b42b04a7b8abe63e8241d0ddc)

Conflicts:
src/mds/SessionMap.cc
src/mds/SessionMap.h

6 years agomds: get ref to message for lambda 26661/head
Patrick Donnelly [Tue, 26 Feb 2019 20:27:40 +0000 (12:27 -0800)]
mds: get ref to message for lambda

Otherwise we have invalid reads. Caught by valgrind.

This cannot be cherry-picked as it was a bug introduced by a backport.

Introduced-by: 2f6289ff7f7385e462d59e096634a1dee0b60837
Fixes: http://tracker.ceph.com/issues/38488
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26546 from cbodley/wip-38410-luminous
Yuri Weinstein [Mon, 4 Mar 2019 17:06:17 +0000 (09:06 -0800)]
Merge pull request #26546 from cbodley/wip-38410-luminous

luminous: rgw: fix cls_bucket_head result order consistency

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26548 from cbodley/wip-38414-luminous
Yuri Weinstein [Mon, 4 Mar 2019 17:05:50 +0000 (09:05 -0800)]
Merge pull request #26548 from cbodley/wip-38414-luminous

luminous: rgw: bug in versioning concurrent, list and get have consistency issue

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25825 from pdvian/wip-36434-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:35:25 +0000 (11:35 -0800)]
Merge pull request #25825 from pdvian/wip-36434-luminous

luminous: mon: monstore tool rebuild does not generate creating_pgs

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25855 from ashishkumsingh/wip-37825-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:30:16 +0000 (11:30 -0800)]
Merge pull request #25855 from ashishkumsingh/wip-37825-luminous

luminous: os/bluestore: KernelDevice::read() does the EIO mapping now.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #25956 from xiexingguo/wip-liminous-async-fixes
Yuri Weinstein [Thu, 28 Feb 2019 19:29:28 +0000 (11:29 -0800)]
Merge pull request #25956 from xiexingguo/wip-liminous-async-fixes

luminous: msg/async: backport recent messenger fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26042 from trociny/wip-37972-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:24:46 +0000 (11:24 -0800)]
Merge pull request #26042 from trociny/wip-37972-luminous

luminous: msg/msg_types: fix the dencoder of entity_addr_t

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26284 from smithfarm/wip-38140-luminous
Yuri Weinstein [Thu, 28 Feb 2019 19:24:17 +0000 (11:24 -0800)]
Merge pull request #26284 from smithfarm/wip-38140-luminous

luminous: tests: Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #26501 from k0ste/luminous_backports2
Yuri Weinstein [Thu, 28 Feb 2019 19:23:50 +0000 (11:23 -0800)]
Merge pull request #26501 from k0ste/luminous_backports2

luminous: mgr/balancer: restrict automatic balancing to specific weekdays

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoceph-volume: use our own testinfra suite for functional testing
Andrew Schoen [Wed, 27 Feb 2019 21:14:03 +0000 (15:14 -0600)]
ceph-volume: use our own testinfra suite for functional testing

This allows us to run a smaller subset of tests from the ones provided
by ceph-ansible and test only the things that pertain to ceph-volume.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ccfc8e6dc42b55f14fa3b4ef210cb639b9a7e20b)

6 years agoMerge pull request #26656 from jan--f/c-v-simple-activate-all-luminous
Andrew Schoen [Wed, 27 Feb 2019 21:58:51 +0000 (15:58 -0600)]
Merge pull request #26656 from jan--f/c-v-simple-activate-all-luminous

luminous: ceph-volume: add --all flag to simple activate

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26065 from pdvian/wip-37989-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:18:17 +0000 (14:18 -0800)]
Merge pull request #26065 from pdvian/wip-37989-luminous

luminous: MDSMonitor: missing osdmon writeable check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25858 from ashishkumsingh/wip-37823-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:14:18 +0000 (14:14 -0800)]
Merge pull request #25858 from ashishkumsingh/wip-37823-luminous

luminous: mds: log evicted clients to clog/dbg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25979 from pdvian/wip-37908-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:13:47 +0000 (14:13 -0800)]
Merge pull request #25979 from pdvian/wip-37908-luminous

luminous: mds: wait shorter intervals if beacon not sent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26215 from pdvian/wip-38102-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:11:50 +0000 (14:11 -0800)]
Merge pull request #26215 from pdvian/wip-38102-luminous

luminous: mds: trim cache after journal flush

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26293 from pdvian/wip-38190-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:11:21 +0000 (14:11 -0800)]
Merge pull request #26293 from pdvian/wip-38190-luminous

luminous: mds: broadcast quota message to client when disable quota

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26473 from pdvian/wip-38336-luminous
Yuri Weinstein [Tue, 26 Feb 2019 22:10:51 +0000 (14:10 -0800)]
Merge pull request #26473 from pdvian/wip-38336-luminous

luminous: mds: fix potential re-evaluate stray dentry in _unlink_local_finish

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoceph-volume: add --all flag to simple activate 26656/head
Jan Fajerski [Thu, 31 Jan 2019 12:45:46 +0000 (13:45 +0100)]
ceph-volume: add --all flag to simple activate

This is intended to behave similarly to the lvm activate --all argument.
When passed, c-v will scan /etc/ceph/osd/ (or the location specified by
CEPH_VOLUME_SIMPLE_JSON_DIR) for json files (glob *.json) and call activate
for each file.  This should greatly ease the take-over of ceph-disk OSDs
with manual commands and deployment tools like DeepSea and ceph-ansible.
Also adds a simple unit test and modifies the
simple/centos7/filestore/activate functional test to use --all.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05af94c6fca66083107e96e9aa1db87619e77e66)

 Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
        resolved by deleting everything after conflict marker

6 years agoMerge pull request #25611 from pdvian/wip-37561-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:47:05 +0000 (12:47 -0800)]
Merge pull request #25611 from pdvian/wip-37561-luminous

luminous: rgw: rgwgc: process coredump in some special case

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25986 from ashishkumsingh/wip-37937-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:46:44 +0000 (12:46 -0800)]
Merge pull request #25986 from ashishkumsingh/wip-37937-luminous

luminous: librgw: export multitenancy support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26310 from pdvian/wip-38193-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:46:20 +0000 (12:46 -0800)]
Merge pull request #26310 from pdvian/wip-38193-luminous

luminous: Object can still be deleted even if s3:DeleteObject policy is set

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25738 from pdvian/wip-37648-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:43:57 +0000 (12:43 -0800)]
Merge pull request #25738 from pdvian/wip-37648-luminous

luminous: rgw: feature -- log successful bucket resharding events

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25901 from ashishkumsingh/wip-37559-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:43:20 +0000 (12:43 -0800)]
Merge pull request #25901 from ashishkumsingh/wip-37559-luminous

luminous: rgw: handle S3 version 2 pre-signed urls with meta-data

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #26131 from pdvian/wip-38038-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:42:40 +0000 (12:42 -0800)]
Merge pull request #26131 from pdvian/wip-38038-luminous

luminous: rgw: fails to start on Fedora 28 from default configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26192 from cbodley/wip-38079-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:42:08 +0000 (12:42 -0800)]
Merge pull request #26192 from cbodley/wip-38079-luminous

luminous: rgw: bucket full sync handles delete markers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26263 from pdvian/wip-38148-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:41:20 +0000 (12:41 -0800)]
Merge pull request #26263 from pdvian/wip-38148-luminous

luminous: rgw: "radosgw-admin bucket rm ... --purge-objects" can hang

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #26332 from smithfarm/wip-38232-luminous
Yuri Weinstein [Mon, 25 Feb 2019 20:40:36 +0000 (12:40 -0800)]
Merge pull request #26332 from smithfarm/wip-38232-luminous

luminous: rgw: fix radosgw linkage with WITH_RADOSGW_BEAST_FRONTEND=OFF

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agomgr/balancer: make weekday be compatible with C 26501/head
xie xingguo [Tue, 19 Feb 2019 10:36:25 +0000 (18:36 +0800)]
mgr/balancer: make weekday be compatible with C

The __desc__ field of **begin[end]_weekend** suggests
sunday is the prevailing start of the week, whereas the
Python implementation says no.

See https://stackoverflow.com/questions/21045569/why-is-pythons-weekday-different-from-tm-wday-in-c

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit b9da0925490d4e9e964249c54e7775d9c76b1719)

6 years agomgr/balancer: restrict automatic balancing to specific weekdays
xie xingguo [Fri, 15 Feb 2019 05:35:53 +0000 (13:35 +0800)]
mgr/balancer: restrict automatic balancing to specific weekdays

We have implemented similar semantics for the scrub behaviour.
Let's try to keep pace with that.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 0412a51ac6dfecc1a72b654196ecb1a599dcc94d)

Conflicts:
- path: src/pybind/mgr/balancer/module.py
  comment: 'get_module_option()' -> 'get_config()' for mgr config-key
    'begin_weekday' and 'begin_weekday' should be int, but
    luminous/mimic config-key always return string.

6 years agoMerge pull request #26587 from smithfarm/wip-doc-fix-luminous
Nathan Cutler [Fri, 22 Feb 2019 13:00:37 +0000 (14:00 +0100)]
Merge pull request #26587 from smithfarm/wip-doc-fix-luminous

luminous: doc: Fix default value of 'mon_osd_max_split_count'

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agodoc: Fix default value of 'mon_osd_max_split_count' 26587/head
Kai Wagner [Fri, 22 Feb 2019 10:25:06 +0000 (11:25 +0100)]
doc: Fix default value of 'mon_osd_max_split_count'

This commit is not cherry-picked from master because the config
parameter mon_osd_max_split_count has been dropped in master

Signed-off-by: Kai Wagner <kwagner@suse.com>
(cherry picked from commit 17d503f5adfbfd96502877bcb55c42ba1eb34a5c)

6 years agocommon/ceph_timer: stop timer's thread when it is suspended 26579/head
Peng Wang [Sun, 17 Feb 2019 09:47:52 +0000 (17:47 +0800)]
common/ceph_timer: stop timer's thread when it is suspended

otherwise the client could hang when shutting down

Fixes: https://tracker.ceph.com/issues/37766
Signed-off-by: Peng Wang <wp389@126.com>
(cherry picked from commit a942722c5bffbacc8d30643dd9863ef844f0617e)

6 years agoMerge pull request #24906 from tchaikov/wip-23673-luminous
Yuri Weinstein [Thu, 21 Feb 2019 23:29:07 +0000 (15:29 -0800)]
Merge pull request #24906 from tchaikov/wip-23673-luminous

luminous: auth: ceph auth add does not sanity-check caps

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25711 from ashishkumsingh/wip-37756-luminous
Yuri Weinstein [Thu, 21 Feb 2019 23:27:41 +0000 (15:27 -0800)]
Merge pull request #25711 from ashishkumsingh/wip-37756-luminous

luminous: osd/PrimaryLogPG: fix the extent length error of the sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25722 from ashishkumsingh/wip-37690-luminous
Yuri Weinstein [Thu, 21 Feb 2019 23:26:41 +0000 (15:26 -0800)]
Merge pull request #25722 from ashishkumsingh/wip-37690-luminous

luminous: tools: ceph-objectstore-tool: Dump hashinfo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25851 from ashishkumsingh/wip-37815-luminous
Yuri Weinstein [Thu, 21 Feb 2019 23:25:38 +0000 (15:25 -0800)]
Merge pull request #25851 from ashishkumsingh/wip-37815-luminous

luminous: qa/workunits/rados/test_health_warnings: prevent out osds

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25853 from ashishkumsingh/wip-37833-luminous
Yuri Weinstein [Thu, 21 Feb 2019 23:24:52 +0000 (15:24 -0800)]
Merge pull request #25853 from ashishkumsingh/wip-37833-luminous

luminous: objecter: avoid race when reset down osd's session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26305 from dzafman/wip-38207
Yuri Weinstein [Thu, 21 Feb 2019 23:23:42 +0000 (15:23 -0800)]
Merge pull request #26305 from dzafman/wip-38207

luminous: mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26078 from ashishkumsingh/wip-37993-luminous
Yuri Weinstein [Thu, 21 Feb 2019 21:54:12 +0000 (13:54 -0800)]
Merge pull request #26078 from ashishkumsingh/wip-37993-luminous

luminous: osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26148 from ashishkumsingh/wip-38046-luminous
Yuri Weinstein [Thu, 21 Feb 2019 21:53:43 +0000 (13:53 -0800)]
Merge pull request #26148 from ashishkumsingh/wip-38046-luminous

luminous: tests: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26236 from pdvian/wip-38105-luminous
Yuri Weinstein [Thu, 21 Feb 2019 21:53:16 +0000 (13:53 -0800)]
Merge pull request #26236 from pdvian/wip-38105-luminous

luminous: osd/PG.cc: account for missing set irrespective of last_complete

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26261 from pdvian/wip-38142-luminous
Yuri Weinstein [Thu, 21 Feb 2019 21:52:38 +0000 (13:52 -0800)]
Merge pull request #26261 from pdvian/wip-38142-luminous

luminous: os/bluestore: fixup access a destroy cond cause deadlock or undefine

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agorgw: bug in versioning concurrent, list and get have consistency issue 26548/head
wanghao72 [Wed, 30 Jan 2019 07:18:06 +0000 (15:18 +0800)]
rgw: bug in versioning concurrent, list and get have consistency issue

Fixes: https://tracker.ceph.com/issues/38060
Signed-off-by: Wang Hao <wanghao72@baidu.com>
(cherry picked from commit cc2a96a47fd71dc8399e31f1a9e87d8a13ea3419)

6 years agorgw: fix cls_bucket_head result order consistency 26546/head
Tianshan Qu [Fri, 3 Nov 2017 07:53:41 +0000 (15:53 +0800)]
rgw: fix cls_bucket_head result order consistency

headers's key is string but bucket_instance_ids's key is int
looked around and found no one use header's key, so change
header from map to vector

Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 46657a0f72530f50f1c94374a27edb57deb70de0)

Conflicts:
src/rgw/rgw_rados.h: unordered bucket listing

6 years agoos/bluestore: simplify Onode tracking for removed object 26540/head
Sage Weil [Wed, 15 Nov 2017 17:02:33 +0000 (11:02 -0600)]
os/bluestore: simplify Onode tracking for removed object

Make the only caller of removed() not need to call note_modified_object
separately, dropping the unneeded erase() call.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3ccc436f9f0d4b010db0c2db15822f8ee46f4873)

6 years agoos/bluestore: keep ref to deleted onode in _do_remove
Sage Weil [Tue, 10 Oct 2017 19:00:30 +0000 (14:00 -0500)]
os/bluestore: keep ref to deleted onode in _do_remove

We need to make sure we carry this ref through until the object is
deleted or else another request right try to read it before the kv
txn is applied.  (This is easy to trigger now that onreadable is completed
at queue time instead of commit time.)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0347518d02eda4e0e2da5241f1d77bc7304d59fb)

6 years agocrush: fix upmap overkill 26127/head
xie xingguo [Sat, 19 Jan 2019 09:19:10 +0000 (17:19 +0800)]
crush: fix upmap overkill

It appears that OSDMap::maybe_remove_pg_upmaps's sanity checks
are overzealous. With some customized crush rules it is possible
for osdmaptool to generate valid upmaps, but maybe_remove_pg_upmaps
will cancel them.

Fixes: http://tracker.ceph.com/issues/37968
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 5c4d241c7f796cb685e9944bf237028162122725)

Conflicts:
- maybe_remove_pg_upmaps input changing
        - slight c++11 auto conflicts

6 years agoosd/OSDMap: using std::vector::reserve to reduce memory reallocation
xie xingguo [Mon, 18 Feb 2019 07:40:22 +0000 (15:40 +0800)]
osd/OSDMap: using std::vector::reserve to reduce memory reallocation

In C++ vectors are dynamic arrays.
Vectors are assigned memory in blocks of contiguous locations.
When the memory allocated for the vector falls short of storing
new elements, a new memory block is allocated to vector and all
elements are copied from the old location to the new location.
This reallocation of elements helps vectors to grow when required.
However, it is a costly operation and time complexity is involved
in this step is linear.
Try to use std::vector::reserve whenever possible if performance
matters.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4a0eabb3a65107cbee5e692ade564102e2b2f8aa)

6 years agoosd/OSDMap: more improvements to upmap
xie xingguo [Sat, 26 Jan 2019 10:03:15 +0000 (18:03 +0800)]
osd/OSDMap: more improvements to upmap

- add ability of appending a 2nd, 3rd, etc... pair to existing upmaps
  when possible, rather than just continuing to the next PG
- handle the underfull case: we can rm-pg-upmap-items if there exist
  any upmaps which remapped a PG out from an underfull OSD

See-also: http://tracker.ceph.com/issues/37940
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a7d2adf4283cc11ed567b0ad07c4076a50d7d2a0)

Conflicts:
- various option changes since luminous
- s/assert/ceph_assert/
        - s/conf./conf->/

6 years agoMerge pull request #25720 from ashishkumsingh/wip-36692-luminous
Yuri Weinstein [Mon, 18 Feb 2019 17:06:25 +0000 (09:06 -0800)]
Merge pull request #25720 from ashishkumsingh/wip-36692-luminous

luminous: rbd-mirror: update mirror status when stopping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25768 from ashishkumsingh/wip-26913-luminous
Yuri Weinstein [Mon, 18 Feb 2019 17:05:42 +0000 (09:05 -0800)]
Merge pull request #25768 from ashishkumsingh/wip-26913-luminous

luminous: mgr: "balancer execute" only requires read permissions

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #25955 from pdvian/wip-37901-luminous
Yuri Weinstein [Mon, 18 Feb 2019 17:04:50 +0000 (09:04 -0800)]
Merge pull request #25955 from pdvian/wip-37901-luminous

luminous: [journal] max journal order is incorrectly set at 64

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #26000 from pdvian/wip-37926-luminous
Yuri Weinstein [Mon, 18 Feb 2019 17:04:19 +0000 (09:04 -0800)]
Merge pull request #26000 from pdvian/wip-37926-luminous

luminous: common: fix for broken rbdmap parameter parsing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #26286 from smithfarm/wip-38186-luminous
Yuri Weinstein [Mon, 18 Feb 2019 17:03:49 +0000 (09:03 -0800)]
Merge pull request #26286 from smithfarm/wip-38186-luminous

luminous: tests: krbd discard no longer guarantees zeroing

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
6 years agomds: fix potential re-evaluate stray dentry in _unlink_local_finish 26473/head
Zhi Zhang [Tue, 12 Feb 2019 02:44:31 +0000 (10:44 +0800)]
mds: fix potential re-evaluate stray dentry in _unlink_local_finish

If MDS handles cap release before _unlink_local_finish, then when MDS
processes _unlink_local_finish, eval_stray might be re-entered twice,
so this will cause crashing.

Fixes: http://tracker.ceph.com/issues/38263
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 72c93baac64211e9f02fa01e743451bf5d63d7a6)

Conflicts:
src/mds/MDCache.h : Resolved in notify_stray

6 years agotest: Add "slow request" to whitelist for Luminous 26686/head
David Zafman [Fri, 15 Feb 2019 10:00:52 +0000 (02:00 -0800)]
test: Add "slow request" to whitelist for Luminous
Redo backport 0e987665fe480530a0d1664392604713828bfa5c

Replaces: 77258ed181dbe04e56da249a71edd3220baf954a

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0e987665fe480530a0d1664392604713828bfa5c)

Conflicts:
qa/suites/rados/singleton/all/osd-recovery.yaml (Use "slow request" for luminous)
qa/suites/rados/singleton/all/thrash-eio.yaml (Use "slow request" for luminous, keep OBJECT_MISPLACED)

6 years agoRevert "qa/suites/rados/singleton: more whitelist"
David Zafman [Fri, 15 Feb 2019 09:59:37 +0000 (01:59 -0800)]
Revert "qa/suites/rados/singleton: more whitelist"

This reverts commit 77258ed181dbe04e56da249a71edd3220baf954a.

Signed-off-by: David Zafman <dzafman@redhat.com>
Conflicts:
qa/suites/rados/singleton/all/osd-recovery.yaml (trivial)
qa/suites/rados/singleton/all/thrash-eio.yaml (trivial)

6 years agoqa/workunits/cephtool/test.sh: Use :- to handle TMPDIR=""
David Zafman [Fri, 15 Feb 2019 07:12:36 +0000 (23:12 -0800)]
qa/workunits/cephtool/test.sh: Use :- to handle TMPDIR=""

Only Luminous has seen TMPDIR="" coming into this script.

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agorestful: Fix regression when traversing leaf nodes 26199/head
Boris Ranto [Thu, 14 Feb 2019 09:35:56 +0000 (10:35 +0100)]
restful: Fix regression when traversing leaf nodes

The commit 23b6c90 introduced a regression when traversing leaf nodes.
The issue is that it traverses the keys of a `dict` returned by
`nodes_by_id`, not the actual `items` of the node. That resulted in an
500 error because it tried to treat `str` as a `dict` and failed.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit fb51bad963cc649fb12ca0ed8b738a2e819299b6)

6 years agomsg/async: do not force updating rotating keys inline 25956/head
xie xingguo [Tue, 15 Jan 2019 05:49:40 +0000 (13:49 +0800)]
msg/async: do not force updating rotating keys inline

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomsg/async: fix mark_down vs accept race
xie xingguo [Tue, 15 Jan 2019 05:47:09 +0000 (13:47 +0800)]
msg/async: fix mark_down vs accept race

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #25990 from ukernel/luminous-37760
Yuri Weinstein [Tue, 12 Feb 2019 21:52:53 +0000 (13:52 -0800)]
Merge pull request #25990 from ukernel/luminous-37760

luminous: mds: handle fragment notify race

Reviewed-by: Venky Shankar <vshankar@redhat.com>