]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
14 years agotest_rados_sync: add import/export test
Colin Patrick McCabe [Fri, 6 May 2011 23:03:43 +0000 (16:03 -0700)]
test_rados_sync: add import/export test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: complain if import src is inaccessible
Colin Patrick McCabe [Fri, 6 May 2011 23:01:02 +0000 (16:01 -0700)]
rados_sync: complain if import src is inaccessible

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: fix --create for import
Colin Patrick McCabe [Fri, 6 May 2011 22:55:55 +0000 (15:55 -0700)]
rados_sync: fix --create for import

Fix import's pool creation faculty.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoadd test_rados_sync.sh
Colin Patrick McCabe [Fri, 6 May 2011 22:55:42 +0000 (15:55 -0700)]
add test_rados_sync.sh

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: fix import a bit
Colin Patrick McCabe [Fri, 6 May 2011 22:18:02 +0000 (15:18 -0700)]
rados_sync: fix import a bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: nicer error formatting
Colin Patrick McCabe [Fri, 6 May 2011 22:11:44 +0000 (15:11 -0700)]
rados_sync: nicer error formatting

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: misc bugfixes
Colin Patrick McCabe [Fri, 6 May 2011 21:47:43 +0000 (14:47 -0700)]
rados_sync: misc bugfixes

Avoid using a temporary c_str pointer when it goes out of scope.

Complain if XATTR_FULLNAME is empty, as well as missing.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: nicer printouts
Colin Patrick McCabe [Fri, 6 May 2011 21:37:55 +0000 (14:37 -0700)]
rados_sync: nicer printouts

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: do_export: fix diff'ing
Colin Patrick McCabe [Fri, 6 May 2011 21:28:02 +0000 (14:28 -0700)]
rados_sync: do_export: fix diff'ing

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: more improvements to xattr_test
Colin Patrick McCabe [Fri, 6 May 2011 21:22:21 +0000 (14:22 -0700)]
rados_sync: more improvements to xattr_test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorados_sync: Add xattr_test
Colin Patrick McCabe [Fri, 6 May 2011 21:13:49 +0000 (14:13 -0700)]
rados_sync: Add xattr_test

Add xattr_test to test if the underlying filesystem supports xattrs

Fix BackedUpObject::get_fs_path a bit.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoCreate rados_sync tool to back up rados pools
Colin Patrick McCabe [Fri, 6 May 2011 19:05:17 +0000 (12:05 -0700)]
Create rados_sync tool to back up rados pools

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: remove misguided strip_owner
Colin Patrick McCabe [Thu, 5 May 2011 18:50:29 +0000 (11:50 -0700)]
obsync: remove misguided strip_owner

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agolibceph: initialize keyring after conf is parsed
Sage Weil [Thu, 5 May 2011 20:07:06 +0000 (13:07 -0700)]
libceph: initialize keyring after conf is parsed

Init the keyring in ceph_mount, after the user has a change to load a conf
file or parse command line args.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoosdmap: small cleanup
Sage Weil [Thu, 5 May 2011 15:51:30 +0000 (08:51 -0700)]
osdmap: small cleanup

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agofix some shadowing arguments
Sage Weil [Wed, 4 May 2011 20:55:50 +0000 (13:55 -0700)]
fix some shadowing arguments

Small subset of what -Wshadow turns up.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agocrypto: fix non-reentrancy of ceph::crypto::init
Colin Patrick McCabe [Thu, 5 May 2011 19:44:40 +0000 (12:44 -0700)]
crypto: fix non-reentrancy of ceph::crypto::init

This could be called multiple times from common_preinit.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agotest-obsync: use s3-tests configuration file
Colin Patrick McCabe [Thu, 5 May 2011 18:39:57 +0000 (11:39 -0700)]
test-obsync: use s3-tests configuration file

Use the same configuration file format as s3-tests. This should make it
easier for people to run obsync tests!

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: remove omit_owner for acls
Colin Patrick McCabe [Thu, 5 May 2011 17:22:48 +0000 (10:22 -0700)]
obsync: remove omit_owner for acls

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: implement --force option
Colin Patrick McCabe [Thu, 5 May 2011 17:18:47 +0000 (10:18 -0700)]
obsync: implement --force option

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: be more verbose about set_xml_acl errors
Colin Patrick McCabe [Thu, 5 May 2011 17:17:57 +0000 (10:17 -0700)]
obsync: be more verbose about set_xml_acl errors

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoboto_tool: add getacl, setacl
Colin Patrick McCabe [Thu, 5 May 2011 00:51:40 +0000 (17:51 -0700)]
boto_tool: add getacl, setacl

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: Fix ACL parsing yet again
Colin Patrick McCabe [Wed, 4 May 2011 21:28:50 +0000 (14:28 -0700)]
obsync: Fix ACL parsing yet again

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorgw: don't output optional DisplayName if not set
Colin Patrick McCabe [Thu, 5 May 2011 00:23:44 +0000 (17:23 -0700)]
rgw: don't output optional DisplayName if not set

Don't output optional DisplayName if not set. Be a little more explicit
about zeroing other optional fields during xml parsing.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorgw: don't assume XML is valid
Colin Patrick McCabe [Wed, 4 May 2011 22:41:36 +0000 (15:41 -0700)]
rgw: don't assume XML is valid

XML parsing functions in RGW now return a bool, indicating whether they
were able to get the fields they needed to out of the XML.
If any field returns false, the parsing is deemed to have failed.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agouclient: fill in InodeCap::wanted member on send_cap.
Greg Farnum [Wed, 4 May 2011 17:50:23 +0000 (10:50 -0700)]
uclient: fill in InodeCap::wanted member on send_cap.

Previously it was never used, which I think led to spamming the MDS
with cap messages!

Hopefully this won't reveal any new cap-handling bugs...

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
14 years agouclient: only try to update caps on the auth MDS.
Greg Farnum [Wed, 4 May 2011 17:47:43 +0000 (10:47 -0700)]
uclient: only try to update caps on the auth MDS.

Previously we would send updates on things like the max_size we
wanted to the first MDS in our list, which was bad if the auth mds
had a higher number. Now, only send them (and update bookkeeping)
for the auth MDS.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
14 years agoobsync: improve ACL handling again
Colin Patrick McCabe [Tue, 3 May 2011 22:50:41 +0000 (15:50 -0700)]
obsync: improve ACL handling again

LocalAcl: store ACLs in memory. They're very small, and the tempfile
stuff was just getting cumbersome.

LocalAcl.equal: do deep comparison of ACLs, not just seeing if the XML
is the same.

Strip owner when setting an ACL. We never want to try to set the owner.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: fix some bugs
Colin Patrick McCabe [Tue, 3 May 2011 22:33:39 +0000 (15:33 -0700)]
obsync: fix some bugs

* Allow parsing of AclPolicy elements that don't have owner_id fields.

* Put all XML attributes in the correct namespace when generating ACL XML

* Fix FileStore.get_acl

* test-obsync: better formatting for obsync command lines

* test-obsync: test ACL-syncing stuff

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: check if ACLs match
Colin Patrick McCabe [Tue, 3 May 2011 20:45:14 +0000 (13:45 -0700)]
obsync: check if ACLs match

If all other aspects of two objects match, we should make sure the ACLs
match before deciding that there's nothing for us to do.

Restructure LocalCopy so that it no longer contains the ACL. Create
LocalAcl to represent a cached local copy of the ACL. Add get_acl
methods to all stores.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoosd: fix pg log entry types to not always be delete
Sage Weil [Tue, 3 May 2011 20:08:35 +0000 (13:08 -0700)]
osd: fix pg log entry types to not always be delete

This was broken by the osd_trans work merged in 01f3526b62.  We need to
use the obs reference to new_obs.  This caused objects to be deleted during
pg recovery.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomon: add 'ceph osd rm N...' command
Sage Weil [Tue, 3 May 2011 19:37:33 +0000 (12:37 -0700)]
mon: add 'ceph osd rm N...' command

So we can mark an old osd as deleted and have it not appear in the osdmap
dump, summary count.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoMerge remote branch 'origin/stable'
Sage Weil [Tue, 3 May 2011 19:36:16 +0000 (12:36 -0700)]
Merge remote branch 'origin/stable'

Conflicts:
src/mon/OSDMonitor.cc

14 years agoosdmap: allow incremental to represent osd deletion
Sage Weil [Tue, 3 May 2011 19:34:54 +0000 (12:34 -0700)]
osdmap: allow incremental to represent osd deletion

Convert new_down to new_state, with values xored onto the old state.  We
preserve compatibility with old incrementals because they were (virtually)
always 0, and we can special case that to mean toggle CEPH_OSD_UP.  We
don't really care if clients get new values right.. if they don't clear
the EXISTS flag that doesn't really hurt them.  It's only important that
the monitor get it right.

To ensure that, we rev the monitor internal protocol.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoobjecter: remove useless mark_down code
Sage Weil [Tue, 3 May 2011 19:28:00 +0000 (12:28 -0700)]
objecter: remove useless mark_down code

We already check sessions a bit further down, and this code only worked
when we got incrementals, not full maps.  Take it out.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agotest-obsync: test ACL translation, run unit tests
Colin Patrick McCabe [Tue, 3 May 2011 18:07:17 +0000 (11:07 -0700)]
test-obsync: test ACL translation, run unit tests

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: remove --owner, elide owner from ACL XML
Colin Patrick McCabe [Tue, 3 May 2011 17:58:04 +0000 (10:58 -0700)]
obsync: remove --owner, elide owner from ACL XML

Just omit the owner field from the ACL XML. It is optional anyway.

Don't supply an --owner switch. The owner will always be the same user
that created the object.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: better usage
Colin Patrick McCabe [Tue, 3 May 2011 17:20:30 +0000 (10:20 -0700)]
obsync: better usage

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: only require --owner if --xuser is set
Colin Patrick McCabe [Tue, 3 May 2011 17:06:11 +0000 (10:06 -0700)]
obsync: only require --owner if --xuser is set

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: implement --owner
Colin Patrick McCabe [Tue, 3 May 2011 01:41:31 +0000 (18:41 -0700)]
obsync: implement --owner

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agocfuse: encode/decode dev_t properly
Sage Weil [Tue, 3 May 2011 01:19:32 +0000 (18:19 -0700)]
cfuse: encode/decode dev_t properly

The fuse layer passes through "encoded" dev_t values (probably for
compatibility reasons or something).  I copied the encode/decode methods
from the kernel and encode/decode the st_rdev values where appropriate
(where struct stat is exposed directory or via the fuse_entry_param
struct).

Fixes: #1031
Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoobsync: implement user translation (--xuser)
Colin Patrick McCabe [Mon, 2 May 2011 21:48:27 +0000 (14:48 -0700)]
obsync: implement user translation (--xuser)

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agorgw: fix ACL XML generation
Colin Patrick McCabe [Mon, 2 May 2011 22:57:06 +0000 (15:57 -0700)]
rgw: fix ACL XML generation

Put AccessControlPolicy in the http://s3.amazonaws.com/doc/2006-03-01/
namespace.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoobsync: refactor LocalCopy
Colin Patrick McCabe [Mon, 2 May 2011 21:45:43 +0000 (14:45 -0700)]
obsync: refactor LocalCopy

Combine LocalCopy, S3StoreLocalCopy, and RadosStoreLocalCopy into one
class called LocalCopy.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoFileStore: use proper object names for linking
Greg Farnum [Mon, 2 May 2011 20:47:23 +0000 (13:47 -0700)]
FileStore: use proper object names for linking

They were backward before, which broke EVERYTHING.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
14 years agoMDS: fix handle_client_rename use of path_traverse.
Greg Farnum [Wed, 27 Apr 2011 23:22:33 +0000 (16:22 -0700)]
MDS: fix handle_client_rename use of path_traverse.

It was using the MDS_TRAVERSE_DISCOVERXLOCK flag, which allows
path_traverse to return success if it encounters a NULL dentry. When
we're looking for a source inode, though, that doesn't work out! We
want MDS_TRAVERSE_DISCOVER, which will go away and look for the dentry
on other inodes but requires a linked dentry, not a NULL one.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
14 years agomds: trim non-auth swallowed subtrees during resolve
Sage Weil [Sat, 30 Apr 2011 00:30:45 +0000 (17:30 -0700)]
mds: trim non-auth swallowed subtrees during resolve

Consider:
 - peer auth for /foo
 - ambiguous import /foo/bar
 - peer claims /foo, swallows /foo/bar.
 - disambiguate_imports sees we didn't get /foo/bar, cancels ambiguous
   import.
 -> we are left with /foo/bar (and content) in cache, even tho it is
   non-auth.

Fix by pulling the try_trim_non_auth_subtree() back out of
cancel_ambiguous_import, and trimming the containing subtree in the
disambiguate (resolve completion) case.  (For the journal replay case the
subtree structure is deterministic and no such check is needed.)

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix replay of EFragment rollback
Sage Weil [Sat, 30 Apr 2011 00:23:58 +0000 (17:23 -0700)]
mds: fix replay of EFragment rollback

Remove from the uncommitted list.

Also, make uncommitted list updated unconditional: we need to do it even
if the inode wasn't already in our cache.

Also, journal the rollback with the same signedness as the prepare, so that
the descriptor/map key matches up.  Adjust signs accordingly.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agorgw: log bytes received
Yehuda Sadeh [Sat, 30 Apr 2011 00:17:23 +0000 (17:17 -0700)]
rgw: log bytes received

14 years agorgw: fix some logging problems
Yehuda Sadeh [Fri, 29 Apr 2011 23:14:22 +0000 (16:14 -0700)]
rgw: fix some logging problems

14 years agorgw_admin: dump also user email
Yehuda Sadeh [Fri, 29 Apr 2011 23:08:04 +0000 (16:08 -0700)]
rgw_admin: dump also user email

14 years agotest/ceph_crypto: Check that the shutdown/fork/init trick works for NSS.
Tommi Virtanen [Fri, 29 Apr 2011 21:48:30 +0000 (14:48 -0700)]
test/ceph_crypto: Check that the shutdown/fork/init trick works for NSS.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
14 years agofilestore: fiemap should close the fd
Yehuda Sadeh [Fri, 29 Apr 2011 21:08:57 +0000 (14:08 -0700)]
filestore: fiemap should close the fd

14 years agofilestore: fiemap should close the fd
Yehuda Sadeh [Fri, 29 Apr 2011 21:08:57 +0000 (14:08 -0700)]
filestore: fiemap should close the fd

14 years agocommon, cfuse: Hook into daemonization and shutdown/init NSS.
Tommi Virtanen [Fri, 29 Apr 2011 18:43:17 +0000 (11:43 -0700)]
common, cfuse: Hook into daemonization and shutdown/init NSS.

NSS cannot tolerate forks without this.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
14 years agomsgr, common: Refactor to extract daemonization out of messenger.
Tommi Virtanen [Fri, 29 Apr 2011 18:29:42 +0000 (11:29 -0700)]
msgr, common: Refactor to extract daemonization out of messenger.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
14 years agomsgr: Rename SimpleMessenger::start(daemonize, nonce) to start_with_nonce.
Tommi Virtanen [Fri, 29 Apr 2011 18:17:53 +0000 (11:17 -0700)]
msgr: Rename SimpleMessenger::start(daemonize, nonce) to start_with_nonce.

Otherwise, once we remove daemonize from the prototype,
all the existing ->start(false) calls will be taken
to mean nonce=0.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
14 years agoceph_crypto: Assert that NSS initialization works.
Tommi Virtanen [Fri, 29 Apr 2011 17:07:10 +0000 (10:07 -0700)]
ceph_crypto: Assert that NSS initialization works.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
14 years agocommon_init: create common_init_daemonize
Colin Patrick McCabe [Fri, 29 Apr 2011 18:17:48 +0000 (11:17 -0700)]
common_init: create common_init_daemonize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agoconfig: Update sample config with more examples
Wido den Hollander [Fri, 29 Apr 2011 17:39:04 +0000 (10:39 -0700)]
config: Update sample config with more examples

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
Signed-off-by: Wido den Hollander <wido@widodh.nl>
14 years agocommon_init: set log_file, not log_dir, by default
Colin Patrick McCabe [Fri, 29 Apr 2011 17:23:10 +0000 (10:23 -0700)]
common_init: set log_file, not log_dir, by default

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agocommon_init: don't modify log_per_instance
Colin Patrick McCabe [Fri, 29 Apr 2011 17:21:08 +0000 (10:21 -0700)]
common_init: don't modify log_per_instance

check it in DoutStreambuf instead.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
14 years agomsgr: remove dup .start() call check, remove cruft
Sage Weil [Fri, 29 Apr 2011 16:49:05 +0000 (09:49 -0700)]
msgr: remove dup .start() call check, remove cruft

There is now no ordering constraint wrt the daemonize bits; those can
safely be pulled out.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
14 years agohadoop: cleanups for libceph type update
Jim Schutt [Fri, 29 Apr 2011 15:13:59 +0000 (09:13 -0600)]
hadoop: cleanups for libceph type update

Signed-off-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
14 years agolfn: put lfn outside of user.ceph namesapce
Sage Weil [Thu, 28 Apr 2011 23:01:23 +0000 (16:01 -0700)]
lfn: put lfn outside of user.ceph namesapce

This completely hides the lfn from the ObjectStore interface users.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
14 years agoMerge remote branch 'origin/master' into lfn
Sage Weil [Thu, 28 Apr 2011 22:45:19 +0000 (15:45 -0700)]
Merge remote branch 'origin/master' into lfn

14 years agomdsmap: show mds name in summary
Sage Weil [Thu, 28 Apr 2011 22:55:20 +0000 (15:55 -0700)]
mdsmap: show mds name in summary

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agohadoop: update libceph types
Sage Weil [Thu, 28 Apr 2011 22:52:07 +0000 (15:52 -0700)]
hadoop: update libceph types

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agohypertable: update libceph types
Sage Weil [Thu, 28 Apr 2011 22:52:01 +0000 (15:52 -0700)]
hypertable: update libceph types

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agolibceph: error out if USE_FILE_OFFSET64 not defined
Sage Weil [Thu, 28 Apr 2011 22:49:37 +0000 (15:49 -0700)]
libceph: error out if USE_FILE_OFFSET64 not defined

Otherwise struct dirent will not match user code and badness on readdir
will ensure.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agolfn: don't return ENOENT if it's not lfn in some cases
Yehuda Sadeh [Thu, 28 Apr 2011 22:44:28 +0000 (15:44 -0700)]
lfn: don't return ENOENT if it's not lfn in some cases

14 years agomds: ignore fragment_notify when dft state doesn't match
Sage Weil [Thu, 28 Apr 2011 22:17:18 +0000 (15:17 -0700)]
mds: ignore fragment_notify when dft state doesn't match

In particular, if there is a resolve in there somewhere, we may have found
out about this refragment from the src because they send resolve messages
to all nodes (to resolve ambiguous migrations).  If that's the case we
can ignore the message.

Fixes crash like

2011-04-28 14:30:31.179106 7fe72e325710 -- 10.0.1.252:6805/22158 <== mds2 10.0.1.252:6803/25635 548 ==== fragment_notify(300000000b4#00* 1) v1 ==== 17+0+0 (2192211443 0 0) 0x2c9ec00 con 0x323d140
2011-04-28 14:30:31.179116 7fe72e325710 mds1.cache handle_fragment_notify fragment_notify(300000000b4#00* 1) v1 from mds2
2011-04-28 14:30:31.179149 7fe72e325710 mds1.cache adjust_dir_fragments 00* 1 on [inode 300000000b4 [...2,head] /syn.4114.0/dir.0/dir.0/dir.0/dir.0/dir.0/dir.5/dir.5/ auth{0=1,2=1} fragtree_t(*^2 00*^1) v188 ap=1 f(v0 m2011-04-28 14:23:59.074510 7=7+0) n(v2 rc2011-04-28 14:23:59.074510 15=14+1) (idft mix->lock g=0,2 dirty) (inest mix dirty) (ifile excl dirty) (ixattr excl) (iversion lock) caps={4114=pAsLsXsxFsx/-@1},l=4114(-1) | dirtyscattered dirfrag caps replicated dirty authpin 0x32bec70]
2011-04-28 14:30:31.179182 7fe72e325710 mds1.cache adjust_dir_fragments 00* bits 1 srcfrags 0x3080860,0x378da50 on [inode 300000000b4 [...2,head] /syn.4114.0/dir.0/dir.0/dir.0/dir.0/dir.0/dir.5/dir.5/ auth{0=1,2=1} fragtree_t(*^2 00*^1) v188 ap=1 f(v0 m2011-04-28 14:23:59.074510 7=7+0) n(v2 rc2011-04-28 14:23:59.074510 15=14+1) (idft mix->lock g=0,2 dirty) (inest mix dirty) (ifile excl dirty) (ixattr excl) (iversion lock) caps={4114=pAsLsXsxFsx/-@1},l=4114(-1) | dirtyscattered dirfrag caps replicated dirty authpin 0x32bec70]
2011-04-28 14:30:31.179218 7fe72e325710 mds1.cache  new fragtree is fragtree_t(*^2 00*^1)
mds/MDCache.cc: In function 'void MDCache::adjust_dir_fragments(CInode*, std::list<CDir*, std::allocator<CDir*> >&, frag_t, int, std::list<CDir*, std::allocator<CDir*> >&, std::list<Context*, std::allocator<Context*> >&, bool)', in thread '0x7fe72e325710'
mds/MDCache.cc: 9254: FAILED assert(srcfrags.size() == 1)
 ceph version 0.27-165-gaf908f8 (commit:af908f82924a67be3aeb2767eaa05ba04c145f42)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x53) [0xa5775e]
 2: (MDCache::adjust_dir_fragments(CInode*, std::list<CDir*, std::allocator<CDir*> >&, frag_t, int, std::list<CDir*, std::allocator<CDir*> >&, std::list<Context*, std::allocator<Context*> >&, bool)+0x2dd) [0x888bbf]
 3: (MDCache::adjust_dir_fragments(CInode*, frag_t, int, std::list<CDir*, std::allocator<CDir*> >&, std::list<Context*, std::allocator<Context*> >&, bool)+0x13d) [0x88817d]
 4: (MDCache::handle_fragment_notify(MMDSFragmentNotify*)+0x199) [0x88bac5]
 5: (MDCache::dispatch(Message*)+0x124) [0x8765ea]
 6: (MDS::handle_deferrable_message(Message*)+0x1f5) [0x77a607]
 7: (MDS::_dispatch(Message*)+0x784) [0x77ba90]

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: do not send fragment_notify to <= recovering nodes
Sage Weil [Thu, 28 Apr 2011 22:02:58 +0000 (15:02 -0700)]
mds: do not send fragment_notify to <= recovering nodes

They will get sorted out during rejoin.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix uninint warning on cur
Sage Weil [Thu, 28 Apr 2011 21:53:35 +0000 (14:53 -0700)]
mds: fix uninint warning on cur

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: handle import cancel while logging EImportStart
Sage Weil [Thu, 28 Apr 2011 21:17:49 +0000 (14:17 -0700)]
mds: handle import cancel while logging EImportStart

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoclient: do not send request to mds -1
Sage Weil [Thu, 28 Apr 2011 21:09:52 +0000 (14:09 -0700)]
client: do not send request to mds -1

If we can't find a target, or the chosen target isn't active, wait.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agolfn: set hash and file name constants
Yehuda Sadeh [Thu, 28 Apr 2011 20:57:45 +0000 (13:57 -0700)]
lfn: set hash and file name constants

14 years agoosd: remove warning about max object name length
Yehuda Sadeh [Thu, 28 Apr 2011 20:51:48 +0000 (13:51 -0700)]
osd: remove warning about max object name length

14 years agomds: try_trim_non_auth_subtree on any canceled import (including resolve)
Sage Weil [Thu, 28 Apr 2011 20:44:55 +0000 (13:44 -0700)]
mds: try_trim_non_auth_subtree on any canceled import (including resolve)

We were trimming on journal replay of an import failure, but not on a
canceled ambiguous import during resolve.  Fix that by moving the call into
the helper (and passing a CDir* instead of a dirfrag_t).

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: make trim_non_auth paths complete filepaths (not dnames)
Sage Weil [Thu, 28 Apr 2011 20:34:34 +0000 (13:34 -0700)]
mds: make trim_non_auth paths complete filepaths (not dnames)

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix steal_dentry dir_auth_pins adjustment
Sage Weil [Thu, 28 Apr 2011 20:22:30 +0000 (13:22 -0700)]
mds: fix steal_dentry dir_auth_pins adjustment

Pass down the correct value for dir_auth_pins (dh->auth_pins plus the
inode's auth_pins, but nothing nested beneath the inode).  The CDentry
doesn't track dir auth pins independently, and doesn't really need to.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomon: use tcmalloc
Sage Weil [Thu, 28 Apr 2011 20:08:34 +0000 (13:08 -0700)]
mon: use tcmalloc

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix export_prep trace format
Sage Weil [Thu, 28 Apr 2011 20:00:44 +0000 (13:00 -0700)]
mds: fix export_prep trace format

The prep message includes a spanning tree in the interior of the subtree
that includes all parent inodes of bounding dirfrags.  That used to look
like
df dentry inode (dir dentry inode)*

The code to generate those traces was stopping if the df->ino had already
been included.  The problem was that we may have done the that inode on a
different dirfrag.

Change this to be

df ('-' | ('f' dir | 'd') dentry inode (dir dentry inode)*)

so that we can start with a dentry (already had the dirfrag, same check
as before) or a dirfrag (already had the inode, the new case), or a '-'
(nothing at all).  A single byte is used to indicate which it is and how
to start decoding.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomon: make 'ceph osd (down,out,in) N' take mulitple osd numbers
Sage Weil [Thu, 28 Apr 2011 19:42:23 +0000 (12:42 -0700)]
mon: make 'ceph osd (down,out,in) N' take mulitple osd numbers

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
14 years agolibceph: no _t types
Sage Weil [Thu, 28 Apr 2011 19:34:11 +0000 (12:34 -0700)]
libceph: no _t types

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agolfn: short fn length is constant and accurate
Yehuda Sadeh [Thu, 28 Apr 2011 18:24:50 +0000 (11:24 -0700)]
lfn: short fn length is constant and accurate

also, disabling real hashing for now

14 years agoosd: bump up max object name size
Yehuda Sadeh [Thu, 28 Apr 2011 18:16:17 +0000 (11:16 -0700)]
osd: bump up max object name size

14 years agocrypto: add support for SHA256
Yehuda Sadeh [Thu, 28 Apr 2011 18:15:50 +0000 (11:15 -0700)]
crypto: add support for SHA256

14 years agolibceph: typedef struct mystruct *mystruct_t
Sage Weil [Thu, 28 Apr 2011 18:11:14 +0000 (11:11 -0700)]
libceph: typedef struct mystruct *mystruct_t

Needed to drop the ceph_ prefix on the internal ceph_dir_result_t type
to prevent the ceph_dir_result_t typedef from colliding.

ceph_mount_info to avoid colliding with int ceph_mount().

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoMerge commit 'origin/master' into lfn
Yehuda Sadeh [Thu, 28 Apr 2011 18:04:40 +0000 (11:04 -0700)]
Merge commit 'origin/master' into lfn

14 years agolibceph: include 'struct' in declarations for C compilation
Sage Weil [Thu, 28 Apr 2011 17:37:18 +0000 (10:37 -0700)]
libceph: include 'struct' in declarations for C compilation

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix auth_pin check
Sage Weil [Thu, 28 Apr 2011 16:30:29 +0000 (09:30 -0700)]
mds: fix auth_pin check

The inode only gets an auth_pin if the dirfrag is not a subtree root.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agoconfig: Remove debug output in conf_get
Wido den Hollander [Thu, 28 Apr 2011 13:03:08 +0000 (15:03 +0200)]
config: Remove debug output in conf_get

Signed-off-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
14 years agoMerge branch 'master' of ceph.newdream.net:git/ceph
Sage Weil [Thu, 28 Apr 2011 00:08:49 +0000 (17:08 -0700)]
Merge branch 'master' of ceph.newdream.net:git/ceph

14 years agomds: handle freeze completion delayed by frozen inode
Sage Weil [Thu, 28 Apr 2011 00:07:54 +0000 (17:07 -0700)]
mds: handle freeze completion delayed by frozen inode

We can't complete a freeze_tree if we are not a subtree and the parent
inode is frozen.  If that's the case, we were just doing nothing on the
auth_unpin, but that means the freeze_tree would never complete.

Instead, retake an auth_pin (on behalf of the parent) and release it when
the parent inode unfreezes.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agolfn: replace hash function
Yehuda Sadeh [Wed, 27 Apr 2011 23:32:52 +0000 (16:32 -0700)]
lfn: replace hash function

for some reason crashes when using libnss

14 years agomds: add 'mds debug auth pins' option
Sage Weil [Wed, 27 Apr 2011 23:34:16 +0000 (16:34 -0700)]
mds: add 'mds debug auth pins' option

This counts dirfrag auth_pins and ensure the inode's nested_auth_pins
count is correct.  Helped catch the bug fixed in the previous commit.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: fix nested_auth_pin accounting on refragment
Sage Weil [Wed, 27 Apr 2011 23:33:21 +0000 (16:33 -0700)]
mds: fix nested_auth_pin accounting on refragment

The diri gets an auth_pin on the first frag pin when it is not a subtree
root.  When we are moving dentries between frags during refragment, make
sure we use the adjust_nested_auth_pins method to have one such pin per
fragment.

Carry an auth_pin on the old fragment for the duration to ensure that the
pinning/unpinning as no side-effects.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: maintain dn pinning invariants during freezing for refragmenting
Sage Weil [Wed, 27 Apr 2011 22:52:26 +0000 (15:52 -0700)]
mds: maintain dn pinning invariants during freezing for refragmenting

fragment_mark_and_complete aims to complete the in-cache directory,
mark+pin every dentry, then drop a final auth_pin so that the whole thing
freezes.  The problem is we may not be holding the final auth_pin, and
other dentries may get added (or removed?) between the mark and freeze
stages.

Use the DNPINNEDFRAG dir state bit to maintain the invariant that that
bit is set IFF all dentries are similarly pinned and marked.  Update the
add_*_dentry and remove_dentry methods to do that.

Fix the success path to assert this was true and to clean up(!).  Also
fix the unwind/failure path to assert.

Signed-off-by: Sage Weil <sage@newdream.net>
14 years agomds: freeze fragments during split/merge
Sage Weil [Wed, 27 Apr 2011 22:09:35 +0000 (15:09 -0700)]
mds: freeze fragments during split/merge

Freeze the target fragment(s) before unfreezing the old fragment(s) to
avoid any weird events going off when the unfreeze unauth_pins the dir
inode (in certain cases).  This makes the whole process cleaner and more
symmetrical.

Signed-off-by: Sage Weil <sage@newdream.net>