David Zafman [Fri, 27 Sep 2013 00:42:13 +0000 (17:42 -0700)]
common, os, osd: Use common functions for safe file reading and writing
Add new safe_read_file() and safe_write_file() to update files atomically
Used instead of original OSD::read_meta(), OSD::write_meta() they are based on
Used by read_superblock() and write_superblock()
Used by write_version_stamp() and version_stamp_is_valid()
Fixes: #6422 Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Wed, 18 Sep 2013 01:14:16 +0000 (18:14 -0700)]
osd: Cleanup init()/read_superblock()
Fix error handling in init()
Cleanup read_superblock() by moving unrelated code into init()
Move init() feature upgrade right after compatibility checking
Remove redundant whoami check
Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Fri, 20 Sep 2013 01:54:36 +0000 (18:54 -0700)]
common, os, osd, test, tools: FileStore must work with ghobjects rather than hobjects
Add ghobject_t to hboject.h header
Add constants NO_SHARD/NO_GEN and change gen_t/shard_t
Convert other headers from hobject_t to ghobject_t
Mostly straight hobject_t to ghobject_t for src/os cc files
Fix tools and tests and enable ceph-dencoder
Add filename generation and parsing including unittest addition
Get ceph-filestore-dump to build
Add gen/shard to DBObjectMap::ghobject_key() and update test case
Add CEPH_FS_FEATURE_INCOMPAT_SHARDS new FileStore feature
Add CEPH_OSD_FEATURE_INCOMPAT_SHARDS new osd feature
Fixes: #5862 Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Wed, 25 Sep 2013 16:19:16 +0000 (09:19 -0700)]
os, osd, tools: Add backportable compatibility checking for sharded objects
OSD
New CEPH_OSD_FEATURE_INCOMPAT_SHARDS
FileStore
NEW CEPH_FS_FEATURE_INCOMPAT_SHARDS
Add FSSuperblock with feature CompatSet in it
Store sharded_objects state using CompatSet
Add set_allow_sharded_objects() and get_allow_sharded_objects() to FileStore/ObjectStore
Add read_superblock()/write_superblock() internal filestore functions
ceph_filestore_dump
Add OSDsuperblock to export format
Use CompatSet from OSD code itself in filestore-dump tool
Always check compatibility of OSD features with on-disk features
On import verify compatibility of on-disk features with export data
Bump super_ver due to export format change
Backport: dumpling, cuttlefish
Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Tue, 17 Sep 2013 20:39:57 +0000 (13:39 -0700)]
include: Bug fixes for CompatSet
FeatureSet insert/remove
Use 64-bit arithmetic to allow features past 31
Allow feature 63 by fixing assert in insert
CompatSet::unsupported() bugs
Ignore feature 0 which became illegal
Use 64-bit arithmetic when computing mask
Use id in insert() and to get correct feature name
Use the right map to get name for diff.ro_compat
mon: don't list "--yes-i-really-mean-it" options as required
By default options are required, and that leads to the Ceph CLI making
users add this to their commands before even submitting it to the monitor. We
want them to see and think about the messages generated by the monitor!
MDS: lock out snapshots until after a flag has been set in the MDSMap
This way users can't put snapshots on their clusters unless they explicitly
ask for them and have seen a warning message. We take a bit of the MDSMap
flags in order to do so. The only thing a little weird here is that anybody
who upgrades to this patch who already has snapshots will hit the EPERM
and have to go through the warning, but it doesn't impact existing snapshots
at all so they should be good.
To go along with this, we add "ever_allowed_snaps" and "explicitly_allowed_snaps"
members to the MDSMap, which are default to false and are set to true
when allow_new_snaps is set. Old maps decoded with new code default to true
and false, respectively, so we can tell.
Sage Weil [Fri, 28 Jun 2013 19:51:07 +0000 (12:51 -0700)]
mds/Server: fix LOOKUPSNAP
The current LOOKUPSNAP code path was dereferencing mdr->dn[0] and getting
SEGV. Instead, set the do_lookup arg to false for getattr so that we do
not try to dereference this; for snaps it will always be NULL as we are
constructing a fake namespace for the .snap directory. (This happens in
the path_traverse, the result of which is to set snapid--without any real
dentry).
With this fix, snaptest-0.sh passes.
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
Sage Weil [Wed, 25 Sep 2013 22:47:28 +0000 (15:47 -0700)]
mon/PGMonitor: fix segfault when osdmap and pgmap stats are out of sync
The leader mon is responsible for propagating changes in the osdmap
into the pgmap (e.g., when a pool or osd goes away). If that hasn't
happened yet, the pg_map map will be out of sync with the current set
of pools. We shouldn't crash.
Sage Weil [Wed, 25 Sep 2013 20:37:13 +0000 (13:37 -0700)]
Makefile: don't use srcdir in TESTS
src/Makefile-env.am:31: error: using '$(srcdir)' in TESTS is currently broken: '$(srcdir)/unittest_bufferlist.sh'
src/Makefile.am:1: 'src/Makefile-env.am' included from here
src/Makefile-env.am:31: error: using '$(srcdir)' in TESTS is currently broken: '$(srcdir)/test/encoding/check-generated.sh'
src/Makefile.am:1: 'src/Makefile-env.am' included from here
The substr_of bufferlist method is used to reduce the copies to the
strict minimum. It is both able to split bufferptr when they are larger
than a chunk and to concatenate them when they are smaller.
autoconf: remove get_command_descriptions dependency to gtest
It is not required and a leftover of the time when it was a unit test
which was a mistake to begin with. It has been reported to create
dependencies problems in some cases, when gest is not available at the
time get_command_descriptions is compiled.
Sage Weil [Tue, 24 Sep 2013 18:22:19 +0000 (11:22 -0700)]
osd/ReplicatedPG: respect RWORDERED rados flag
If this flag is set, we need to order reads as writes. In particular, this
means that reads will wait for degraded object recovery even if there is a
local copy. And subsequently will be ordered after a preceding write that
is waiting for the same thing.
The TEST_F(ErasureCodePluginRegistryTest, factory_mutex) was bugous and
only succeeded by chance. The sleep was on the factory constructor which
was never called. An erasure code plugin that hangs forever on load is
created instead.
The sleep_forever.detach is replaced by pthread_cancel to interrupt the
thread that hangs forever. If not, gtest will try to join the thread and
never exit.
projects / ceph.git / log