]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agolibrbd: corrected resize RPC message backwards compatibility 14615/head
Jason Dillaman [Tue, 18 Apr 2017 14:05:53 +0000 (10:05 -0400)]
librbd: corrected resize RPC message backwards compatibility

Commit d1f2c557 incorrectly changed the order of variables within
the payload. This resulted in breaking the resize RPC message
with older versions of Ceph.

Fixes: http://tracker.ceph.com/issues/19636
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14600 from tchaikov/wip-19390
Sage Weil [Tue, 18 Apr 2017 13:37:07 +0000 (08:37 -0500)]
Merge pull request #14600 from tchaikov/wip-19390

debian: package ceph.logroate properly

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
8 years agoMerge pull request #14396 from jcsp/wip-19550
John Spray [Tue, 18 Apr 2017 11:59:14 +0000 (12:59 +0100)]
Merge pull request #14396 from jcsp/wip-19550

qa: re-enable ENOSPC tests for kclient

8 years agoMerge pull request #14589 from jcsp/wip-19640
John Spray [Tue, 18 Apr 2017 11:58:37 +0000 (12:58 +0100)]
Merge pull request #14589 from jcsp/wip-19640

client: refine fsync/close writeback error handling

Reviewed-by: Jeff Layton <jlayton@redhat.com>
8 years agoMerge pull request #14272 from jcsp/wip-vstart-fixup
John Spray [Tue, 18 Apr 2017 11:50:20 +0000 (12:50 +0100)]
Merge pull request #14272 from jcsp/wip-vstart-fixup

qa: fix test_standby_for_invalid_fscid with vstart_runner

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #14196 from jcsp/wip-cephfs-relnotes
John Spray [Tue, 18 Apr 2017 11:50:04 +0000 (12:50 +0100)]
Merge pull request #14196 from jcsp/wip-cephfs-relnotes

PendingReleaseNotes: recent cephfs changes

8 years agoMerge pull request #14105 from jcsp/wip-pretty-tell
John Spray [Tue, 18 Apr 2017 11:49:39 +0000 (12:49 +0100)]
Merge pull request #14105 from jcsp/wip-pretty-tell

mds: pretty json from `tell` commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #14104 from jcsp/wip-18509
John Spray [Tue, 18 Apr 2017 11:48:52 +0000 (12:48 +0100)]
Merge pull request #14104 from jcsp/wip-18509

mds: include advisory `path` field in damage

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoMerge pull request #14164 from jcsp/wip-16842-mitigation
John Spray [Tue, 18 Apr 2017 11:48:20 +0000 (12:48 +0100)]
Merge pull request #14164 from jcsp/wip-16842-mitigation

Mitigation for #16842, validate sessions after load

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoclient: refine fsync/close writeback error handling 14589/head
John Spray [Mon, 17 Apr 2017 12:52:12 +0000 (08:52 -0400)]
client: refine fsync/close writeback error handling

Previously, errors stuck indelibly to the inode, which
meant that a close call would see an error even if the
user already dutifully fsync()'d and handled it.

We should emit each error only once per file handle.

Signed-off-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14585 from optimistyzy/414
Haomai Wang [Tue, 18 Apr 2017 11:23:30 +0000 (19:23 +0800)]
Merge pull request #14585 from optimistyzy/414

bluestore/NVMEDEVICE: update SPDK to version 17.03

Reviewed-by: Haomai Wang <haomai@xsky.com>
8 years agoMerge pull request #12535 from ceph/wip-rgw-multisite-teuthology
Orit Wasserman [Tue, 18 Apr 2017 08:47:48 +0000 (11:47 +0300)]
Merge pull request #12535 from ceph/wip-rgw-multisite-teuthology

rgw: multisite enabled over multiple clusters
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #14555 from yaozongyou/fix-readme-notconsistent
Kefu Chai [Tue, 18 Apr 2017 07:59:23 +0000 (15:59 +0800)]
Merge pull request #14555 from yaozongyou/fix-readme-notconsistent

README.md: fix build instructions inconsistent.

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agodebian: package ceph.logroate properly 14600/head
Kefu Chai [Tue, 18 Apr 2017 06:07:04 +0000 (14:07 +0800)]
debian: package ceph.logroate properly

see also "man dh_installlogrotate"

Fixes: http://tracker.ceph.com/issues/19390
Signed-off-by: Kefu Chai <kchai@redhat.com>
8 years agobluestore/NVMEDEVICE: update SPDK to version 17.03 14585/head
optimistyzy [Fri, 14 Apr 2017 05:38:40 +0000 (13:38 +0800)]
bluestore/NVMEDEVICE: update SPDK to version 17.03

Do some minor changes:

1 Restrict the total DPDK memory used by an osd instance.
change the name from bluestore_spdk_socket_mem to
bluestore_spdk_mem.

2 use spdk_env_init instead of rte_eal_init. The reason is that
SPDK lib invokes rte_eal_init which reduces the initilization
paramter conversion and check, also spdk 17.03 invokes
spdk_vtophys_register_dpdk_mem() (which is an internal function)
in spdk_env_init, and this func must be called.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
8 years agoMerge pull request #14591 from tchaikov/wip-readme-headings
Sage Weil [Mon, 17 Apr 2017 21:43:43 +0000 (16:43 -0500)]
Merge pull request #14591 from tchaikov/wip-readme-headings

README.md: use github heading syntax to mark the headings

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
8 years agoMerge pull request #13968 from dzafman/wip-15912-followon
Sage Weil [Mon, 17 Apr 2017 21:42:13 +0000 (16:42 -0500)]
Merge pull request #13968 from dzafman/wip-15912-followon

osd,mon: misc full fixes and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14440 from liewegas/wip-status-flags
Yuri Weinstein [Mon, 17 Apr 2017 20:34:37 +0000 (13:34 -0700)]
Merge pull request #14440 from liewegas/wip-status-flags

osd/OSDMap: hide require_*_osd and sortbitwise flags

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agotest: Test health check output for full ratios 13968/head
David Zafman [Sat, 15 Apr 2017 00:36:17 +0000 (17:36 -0700)]
test: Test health check output for full ratios

Test out of order ratios summary and details
Test various full osd conditions summary and details

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoMerge pull request #14561 from linuxbox2/wip-rgw-reread-dir
Matt Benjamin [Mon, 17 Apr 2017 18:58:35 +0000 (14:58 -0400)]
Merge pull request #14561 from linuxbox2/wip-rgw-reread-dir

rgw_file: fix readdir after dirent-change

8 years agoREADME.md: use github heading syntax to mark the headings 14591/head
Kefu Chai [Mon, 17 Apr 2017 15:49:32 +0000 (23:49 +0800)]
README.md: use github heading syntax to mark the headings

Signed-off-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14466 from fangyuxiangGL/bi
Casey Bodley [Mon, 17 Apr 2017 15:04:02 +0000 (11:04 -0400)]
Merge pull request #14466 from fangyuxiangGL/bi

rgw: bucket stats display bucket index type

Reviewed-by: liuchang0812 <liuchang0812@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agomon, osd: Add detailed full information for now in the mon
David Zafman [Fri, 14 Apr 2017 03:42:55 +0000 (20:42 -0700)]
mon, osd: Add detailed full information for now in the mon

Show ceph health doc output in the correct order

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agomon: Issue warning or error if a full ratio out of order
David Zafman [Thu, 13 Apr 2017 23:20:27 +0000 (16:20 -0700)]
mon: Issue warning or error if a full ratio out of order

The full ratios should be in this order: nearfull, backfillfull, full, failsafe full

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agomon: Always fix-up full ratios when specified incorrectly in config
David Zafman [Thu, 13 Apr 2017 18:43:18 +0000 (11:43 -0700)]
mon: Always fix-up full ratios when specified incorrectly in config

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agomon: Use currently configure full ratio to determine available space
David Zafman [Thu, 13 Apr 2017 18:41:18 +0000 (11:41 -0700)]
mon: Use currently configure full ratio to determine available space

This is a bug that would not adjust available space based on the
currently configured full ratio, but rather the mon_osd_full_ratio
default initial value.

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: check_full_status() remove bogus comment and use equivalent computation
David Zafman [Wed, 12 Apr 2017 05:04:07 +0000 (22:04 -0700)]
osd: check_full_status() remove bogus comment and use equivalent computation

We actually compute kb_used as the kb - kb_avail.  We don't have the
statfs() system call issue of non-privileged f_bavail vs f_bfree.  It
was assumed that used was really like (blocks - f_bfree).  It is not.

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Check whether any OSD is full before starting recovery
David Zafman [Wed, 5 Apr 2017 21:12:43 +0000 (14:12 -0700)]
osd: Check whether any OSD is full before starting recovery

Add event RecoveryTooFull to move to NotRecovering state

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Add PG state and flag for too full for recovery
David Zafman [Wed, 5 Apr 2017 21:09:18 +0000 (14:09 -0700)]
osd: Add PG state and flag for too full for recovery

New state machine state NotRecovering
New PG state PG_STATE_RECOVERY_TOOFULL

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Add check_osdmap_full() to check for shard OSD fullness
David Zafman [Thu, 16 Mar 2017 15:05:58 +0000 (08:05 -0700)]
osd: Add check_osdmap_full() to check for shard OSD fullness

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Rename backfill_request_* to recovery_request_*
David Zafman [Thu, 16 Mar 2017 17:30:57 +0000 (10:30 -0700)]
osd: Rename backfill_request_* to recovery_request_*

To be used by both recovery and backfill

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Check failsafe full and crash on push/pull
David Zafman [Mon, 3 Apr 2017 23:20:35 +0000 (16:20 -0700)]
osd: Check failsafe full and crash on push/pull

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Revamp injectfull op to support all full states
David Zafman [Fri, 31 Mar 2017 21:13:14 +0000 (14:13 -0700)]
osd: Revamp injectfull op to support all full states

Use check_* for injectable full checks
Use is_* to just test simple cur_state

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Handle backfillfull_ratio just like nearfull and full
David Zafman [Thu, 30 Mar 2017 19:42:54 +0000 (12:42 -0700)]
osd: Handle backfillfull_ratio just like nearfull and full

Add BACKFILLFULL as a local OSD cur_state
Notify monitor of this new fullness state

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: For testing full disks add injectfull socket command
David Zafman [Thu, 30 Mar 2017 18:18:38 +0000 (11:18 -0700)]
osd: For testing full disks add injectfull socket command

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agocommon: Bump ratio for backfillfull from 85% to 90%
David Zafman [Mon, 3 Apr 2017 22:46:37 +0000 (15:46 -0700)]
common: Bump ratio for backfillfull from 85% to 90%

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agocommon: Remove unused config option osd_recovery_threads
David Zafman [Thu, 30 Mar 2017 00:35:51 +0000 (17:35 -0700)]
common: Remove unused config option osd_recovery_threads

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: too_full_for_backfill() returns ostream for reason
David Zafman [Thu, 30 Mar 2017 18:17:13 +0000 (11:17 -0700)]
osd: too_full_for_backfill() returns ostream for reason

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Fail-safe full is a hard stop even for mds
David Zafman [Thu, 30 Mar 2017 17:30:29 +0000 (10:30 -0700)]
osd: Fail-safe full is a hard stop even for mds

We can't allow OSD to become non-startable even if mds
could be writing as part of file removals.

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agotest: Fix intended test flow and restore nearfull-ratio
David Zafman [Fri, 7 Apr 2017 18:52:57 +0000 (11:52 -0700)]
test: Fix intended test flow and restore nearfull-ratio

This is inconsequential but seems to have always been wrong since original
commit 6cafb0e3e0bc5f992c2483a46cb00e83dca035cc

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agotest: Switch from pg to osd for set-*-ratio commands
David Zafman [Fri, 7 Apr 2017 16:36:26 +0000 (09:36 -0700)]
test: Switch from pg to osd for set-*-ratio commands

Testing of 6422e0a220fb3f32ccae50e0c7e52dc9984685c6

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Remove unused argument to clear_queued_recovery
David Zafman [Thu, 16 Mar 2017 17:27:31 +0000 (10:27 -0700)]
osd: Remove unused argument to clear_queued_recovery

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Increase osd_backfill_retry_interval to 30 seconds
David Zafman [Thu, 16 Mar 2017 17:40:08 +0000 (10:40 -0700)]
osd: Increase osd_backfill_retry_interval to 30 seconds

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoceph-objectstore-tool: cleanup comment
David Zafman [Mon, 3 Apr 2017 18:28:42 +0000 (11:28 -0700)]
ceph-objectstore-tool: cleanup comment

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoosd: Fix log message
David Zafman [Tue, 28 Feb 2017 20:28:38 +0000 (12:28 -0800)]
osd: Fix log message

Signed-off-by: David Zafman <dzafman@redhat.com>
8 years agoMerge pull request #12536 from ilc/cls_rgw_minor_refactor
Casey Bodley [Mon, 17 Apr 2017 13:40:47 +0000 (09:40 -0400)]
Merge pull request #12536 from ilc/cls_rgw_minor_refactor

cls/rgw: Clean up the "magic string" usage in the cls layer for RGW.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
8 years agoMerge pull request #14046 from songbaisen/m7
John Spray [Mon, 17 Apr 2017 10:23:22 +0000 (11:23 +0100)]
Merge pull request #14046 from songbaisen/m7

mgr: optimization some judgment and adjust the debug remove value in register_new_pgs

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agomgr: optimization some judgment and adjust the debug remove value in register_new_pgs 14046/head
songbaisen [Mon, 20 Mar 2017 11:31:37 +0000 (19:31 +0800)]
mgr: optimization some judgment and adjust the debug remove value in register_new_pgs

Signed-off-by: song baisen <song.baisen@zte.com.cn>
8 years agoMerge pull request #14267 from yunfeiguan/master
John Spray [Sat, 15 Apr 2017 18:46:53 +0000 (19:46 +0100)]
Merge pull request #14267 from yunfeiguan/master

cephfs: fix mount point break off problem after mds switch occured

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoMerge pull request #13587 from yanghonggang/master
John Spray [Sat, 15 Apr 2017 18:45:54 +0000 (19:45 +0100)]
Merge pull request #13587 from yanghonggang/master

cephfs: fix write_buf's _len overflow problem

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #13522 from singler/wip-reconnect-client
John Spray [Sat, 15 Apr 2017 18:44:53 +0000 (19:44 +0100)]
Merge pull request #13522 from singler/wip-reconnect-client

client/Client.cc: after reset session from MDS - reconnect

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoMerge pull request #13466 from jcsp/wip-cephfs-ecoverwrites
John Spray [Sat, 15 Apr 2017 18:44:00 +0000 (19:44 +0100)]
Merge pull request #13466 from jcsp/wip-cephfs-ecoverwrites

Add test for is_hacky_ecoverwrites in cephfs pool checks

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14207 from jcsp/wip-19401
John Spray [Sat, 15 Apr 2017 18:40:59 +0000 (19:40 +0100)]
Merge pull request #14207 from jcsp/wip-19401

mds: ignore ENOENT on writing backtrace

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agorgw_file: fix readdir after dirent-change 14561/head
Matt Benjamin [Fri, 14 Apr 2017 19:56:37 +0000 (15:56 -0400)]
rgw_file: fix readdir after dirent-change

Also, fixes link count computation off-by-one, update of state.nlink
after computation, link computation reset at start, and a time print
in debug log.

Fixes: http://tracker.ceph.com/issues/19634
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
link count

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agoMerge pull request #13579 from ukernel/wip-19022
John Spray [Sat, 15 Apr 2017 11:15:04 +0000 (12:15 +0100)]
Merge pull request #13579 from ukernel/wip-19022

client: drop cap snaps when auth mds session gets closed

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14490 from liewegas/wip-no-luminous
Sage Weil [Sat, 15 Apr 2017 01:05:45 +0000 (20:05 -0500)]
Merge pull request #14490 from liewegas/wip-no-luminous

mon: add mon_debug_no_require_luminous

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14553 from guihecheng/rgw_file-revert
Matt Benjamin [Fri, 14 Apr 2017 18:30:38 +0000 (14:30 -0400)]
Merge pull request #14553 from guihecheng/rgw_file-revert

rgw_file: fix up potential race condition

verified f23

8 years agoMerge pull request #14546 from dachary/wip-19628-directory-boot
Kefu Chai [Fri, 14 Apr 2017 16:06:50 +0000 (00:06 +0800)]
Merge pull request #14546 from dachary/wip-19628-directory-boot

ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoqa/suites/rados/thrash: do not thrash cluster full without luminous flag 14490/head
Sage Weil [Fri, 14 Apr 2017 16:04:47 +0000 (12:04 -0400)]
qa/suites/rados/thrash: do not thrash cluster full without luminous flag

The interface changes with the luminous flag; this will fail.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoqa/suites/rados: expand other collections with no-require-luminous
Sage Weil [Fri, 14 Apr 2017 15:45:05 +0000 (11:45 -0400)]
qa/suites/rados: expand other collections with no-require-luminous

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14551 from tchaikov/wip-19584
Kefu Chai [Fri, 14 Apr 2017 14:42:34 +0000 (22:42 +0800)]
Merge pull request #14551 from tchaikov/wip-19584

mon/OSDMonitor: transit creating_pgs from pgmap when upgrading

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14435 from liewegas/wip-bluestore-deferred
Sage Weil [Fri, 14 Apr 2017 13:35:50 +0000 (08:35 -0500)]
Merge pull request #14435 from liewegas/wip-bluestore-deferred

os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
8 years agoMerge pull request #14489 from liewegas/wip-bluestore-fix-flush
Sage Weil [Fri, 14 Apr 2017 13:35:35 +0000 (08:35 -0500)]
Merge pull request #14489 from liewegas/wip-bluestore-fix-flush

os/bluestore: fix narrow osr->flush() race

8 years agomon/OSDMonitor: require_luminous_osds for pg-remap[-items]
Sage Weil [Fri, 14 Apr 2017 13:21:17 +0000 (09:21 -0400)]
mon/OSDMonitor: require_luminous_osds for pg-remap[-items]

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoFix build instructions inconsistent. 14555/head
Yao Zongyou [Fri, 14 Apr 2017 12:56:16 +0000 (20:56 +0800)]
Fix build instructions inconsistent.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
8 years agoceph-disk: enable directory backed OSD at boot time 14546/head
Loic Dachary [Thu, 13 Apr 2017 21:49:50 +0000 (23:49 +0200)]
ceph-disk: enable directory backed OSD at boot time

https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6
introduced a regression preventing directory backed OSD from starting at
boot time.

For device backed OSD the boot sequence starts with ceph-disk@.service
and proceeds to

    systemctl enable --runtime ceph-osd@.service

where the --runtime ensure ceph-osd@12 is removed when the machine
reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at
boot time.

However directory backed OSD solely rely on the ceph-osd@.service unit
to start at boot time and will therefore fail to boot.

The --runtime flag is selectively set for device backed OSD only.

Fixes: http://tracker.ceph.com/issues/19628
Signed-off-by: Loic Dachary <loic@dachary.org>
8 years agorgw_file: add locks to fix potential race 14553/head
Gui Hecheng [Fri, 14 Apr 2017 09:28:19 +0000 (17:28 +0800)]
rgw_file: add locks to fix potential race

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
8 years agoMerge pull request #13859 from jcsp/wip-19204
John Spray [Fri, 14 Apr 2017 09:50:14 +0000 (10:50 +0100)]
Merge pull request #13859 from jcsp/wip-19204

mds: shut down finisher before objecter

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoMerge pull request #14505 from jcsp/wip-19566
John Spray [Fri, 14 Apr 2017 09:49:20 +0000 (10:49 +0100)]
Merge pull request #14505 from jcsp/wip-19566

mds: fix mgrc shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #14398 from jcsp/wip-19551
John Spray [Fri, 14 Apr 2017 09:48:25 +0000 (10:48 +0100)]
Merge pull request #14398 from jcsp/wip-19551

mon: emit cluster log messages on MDS health changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoqa: add log whitelists for MDS health messages 14398/head
John Spray [Thu, 13 Apr 2017 16:01:09 +0000 (12:01 -0400)]
qa: add log whitelists for MDS health messages

Now that we send these to the cluster log, we must
whitelist them in the tests that exercise those
unhealthy states.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomon/OSDMonitor: transit creating_pgs from pgmap when upgrading 14551/head
Kefu Chai [Fri, 14 Apr 2017 07:49:14 +0000 (15:49 +0800)]
mon/OSDMonitor: transit creating_pgs from pgmap when upgrading

there could be some pg(s) still being created when we are upgrading to
luminous, and the pools holding them are not changed in the sense of
pg_pool_t::last_change after the upgrade and before we scan for
creating pgs. in that case, the existing update_pending_creatings()
will fail to collect the pgs being created before the upgrade.

with this change, the creating_pgs in pgmap are also used for updating
the OSDMonitor's creating_pgs if it's updated.

but we should stopupdating the pgmap once the upgrade completes. i.e.
stop dispatching MSG_PGSTATS messages to PGMonitor if the quorum and all
osds are luminous.

Fixes: http://tracker.ceph.com/issues/19584
Signed-off-by: Kefu Chai <kchai@redhat.com>
8 years agoRevert "rgw_file.h: fix potential data race condition"
Gui Hecheng [Wed, 12 Apr 2017 08:46:49 +0000 (16:46 +0800)]
Revert "rgw_file.h: fix potential data race condition"

This reverts commit 3f7b411e1e304e8a6f295dc50b8cf3c6c496f4cc.
Causing hung up due to double lock.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
8 years agoMerge pull request #10851 from atheism/rgw-s3-doc-update
Kefu Chai [Fri, 14 Apr 2017 02:18:10 +0000 (10:18 +0800)]
Merge pull request #10851 from atheism/rgw-s3-doc-update

doc/radosgw/s3/cpp.rst: update usage of libs3 APIs to make the examples work

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14469 from badone/wip-return-error-on-missing-client-id
Brad Hubbard [Thu, 13 Apr 2017 22:10:59 +0000 (08:10 +1000)]
Merge pull request #14469 from badone/wip-return-error-on-missing-client-id

mds: Return error message instead of asserting

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agomon/OSDMonitor: slightly better debug output
Sage Weil [Thu, 13 Apr 2017 18:12:42 +0000 (14:12 -0400)]
mon/OSDMonitor: slightly better debug output

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agomon/OSDMonitor: fix creating pg trimming during upgrade
Sage Weil [Thu, 13 Apr 2017 17:30:41 +0000 (13:30 -0400)]
mon/OSDMonitor: fix creating pg trimming during upgrade

The creating pgs are in the PGMap with the CREATING
state bit set.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoqa/suites/rados: require_luminous_osds at end of run, then scrub
Sage Weil [Mon, 10 Apr 2017 19:00:43 +0000 (15:00 -0400)]
qa/suites/rados: require_luminous_osds at end of run, then scrub

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoqa/tasks/exec_on_cleanup.py: add
Sage Weil [Thu, 13 Apr 2017 14:09:26 +0000 (10:09 -0400)]
qa/tasks/exec_on_cleanup.py: add

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14508 from smithfarm/wip-doc-dev
Loic Dachary [Thu, 13 Apr 2017 18:54:46 +0000 (20:54 +0200)]
Merge pull request #14508 from smithfarm/wip-doc-dev

doc: dev guide: how to run s3-tests locally against vstart

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoos/bluestore: tune deferred_batch_ops separately for hdd and ssd 14435/head
Sage Weil [Mon, 10 Apr 2017 14:26:40 +0000 (10:26 -0400)]
os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14420 from optimistyzy/329
Sage Weil [Thu, 13 Apr 2017 17:34:51 +0000 (12:34 -0500)]
Merge pull request #14420 from optimistyzy/329

os/bluestore/NVMEDevice: Add multiple thread support for SPDK I/O thread

Reviewed-by: Haomai Wang <haomai@xsky.com>
8 years agodoc: dev guide: how to run s3-tests locally against vstart 14508/head
Nathan Cutler [Thu, 13 Apr 2017 17:14:52 +0000 (19:14 +0200)]
doc: dev guide: how to run s3-tests locally against vstart

Add a bunch of verbiage to the Developer Guide

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #14458 from linuxbox2/wip-rgw_file-readahead
Matt Benjamin [Thu, 13 Apr 2017 17:12:27 +0000 (13:12 -0400)]
Merge pull request #14458 from linuxbox2/wip-rgw_file-readahead

rgw_file: support readdir cb type hints (plus fixes)

8 years agomds: shut down finisher before objecter 13859/head
John Spray [Wed, 8 Mar 2017 12:13:46 +0000 (12:13 +0000)]
mds: shut down finisher before objecter

Some of the finisher contexts would try to call into Objecter.
We mostly are protected from this by mds_lock+the stopping
flag, but at the Filer level there's no mds_lock, so in the
case of file size probing we have a problem.

Fixes: http://tracker.ceph.com/issues/19204
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #13966 from jcsp/wip-client-logmsg
John Spray [Thu, 13 Apr 2017 16:43:10 +0000 (17:43 +0100)]
Merge pull request #13966 from jcsp/wip-client-logmsg

client: specify inode in get_caps log message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #14087 from jcsp/wip-doc-fscid
John Spray [Thu, 13 Apr 2017 16:42:13 +0000 (17:42 +0100)]
Merge pull request #14087 from jcsp/wip-doc-fscid

doc: how to specify filesystem for cephfs clients

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agomds: include advisory `path` field in damage 14104/head
John Spray [Thu, 23 Mar 2017 13:07:32 +0000 (09:07 -0400)]
mds: include advisory `path` field in damage

This will just be whatever path we were looking
at at the point that damage was notified -- no
intention whatsoever of providing any up to date
path or resolution when there are multiple paths
to an inode.

Fixes: http://tracker.ceph.com/issues/18509
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomds: validate prealloc_inos on sessions after load 14164/head
John Spray [Mon, 27 Mar 2017 11:56:31 +0000 (12:56 +0100)]
mds: validate prealloc_inos on sessions after load

Mitigates http://tracker.ceph.com/issues/16842

Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomds: operator<< for Session
John Spray [Mon, 27 Mar 2017 11:33:59 +0000 (12:33 +0100)]
mds: operator<< for Session

Use this to get a nice human readable name
when available (also including the session id in
parentheses)

Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomds: pretty json from `tell` commands 14105/head
John Spray [Thu, 23 Mar 2017 13:31:41 +0000 (09:31 -0400)]
mds: pretty json from `tell` commands

The overhead of the whitespace is trivial and
makes the output somewhat human readable.  Previously
I was always taking `damage ls` into a file and
parsing it out with python.

Signed-off-by: John Spray <john.spray@redhat.com>
8 years agoPendingReleaseNotes: recent cephfs changes 14196/head
John Spray [Tue, 28 Mar 2017 15:13:51 +0000 (16:13 +0100)]
PendingReleaseNotes: recent cephfs changes

Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomds: ignore ENOENT on writing backtrace 14207/head
John Spray [Tue, 28 Mar 2017 18:13:33 +0000 (14:13 -0400)]
mds: ignore ENOENT on writing backtrace

We get ENOENT when a pool doesn't exist.  This can
happen because we don't prevent people deleting
former cephfs data pools whose files may not have
had their metadata flushed yet.

http://tracker.ceph.com/issues/19401
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agorgw multisite: use get_config_master_client for radosgw_admin task 12535/head
Ali Maredia [Tue, 20 Dec 2016 18:09:12 +0000 (13:09 -0500)]
rgw multisite: use get_config_master_client for radosgw_admin task

Signed-off-by: Ali Maredia <amaredia@redhat.com>
8 years agorgw: multisite enabled over multiple clusters
Ali Maredia [Fri, 16 Dec 2016 18:23:09 +0000 (13:23 -0500)]
rgw: multisite enabled over multiple clusters

Added '--cluster' to all necessary commands
ex: radosgw-admin, rados, ceph, made sure
necessary checks were in place so that clients
can be read with our without a cluster_name
preceeding them

Made master_client defined in the config for
radosgw-admin task

Signed-off-by: Ali Maredia <amaredia@redhat.com>
8 years agorgw_file: don't expire directories being read 14458/head
Matt Benjamin [Tue, 11 Apr 2017 10:42:07 +0000 (06:42 -0400)]
rgw_file: don't expire directories being read

If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agomon: emit cluster log messages on MDS health changes
John Spray [Fri, 7 Apr 2017 13:24:01 +0000 (09:24 -0400)]
mon: emit cluster log messages on MDS health changes

Previously, when we got a beacon that updated the health
metrics for an MDS, the user would just see mysterious-looking
cluster log messages indicating a rising fsmap epoch number.

It would be good to do this for health messages in general at
some point, but for now just do it for the MDS ones.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agomds: fix mgrc shutdown 14505/head
John Spray [Thu, 13 Apr 2017 15:00:46 +0000 (11:00 -0400)]
mds: fix mgrc shutdown

Were previously only tearing MgrClient down when not
holding a rank, leading to it trying to continue
to run after monclient was shut down.

Fixes: http://tracker.ceph.com/issues/19566
Signed-off-by: John Spray <john.spray@redhat.com>
8 years agorgw_file: chunked readdir
Matt Benjamin [Tue, 11 Apr 2017 09:56:13 +0000 (05:56 -0400)]
rgw_file:  chunked readdir

Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agorgw_file: introduce rgw_lookup type hints
Matt Benjamin [Wed, 5 Apr 2017 00:16:13 +0000 (20:16 -0400)]
rgw_file: introduce rgw_lookup type hints

The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>