Samuel Just [Mon, 24 Sep 2012 21:33:17 +0000 (14:33 -0700)]
PG: explicitely delay ops on backfill_pos
Previously, we considered backfill_pos degraded in order to delay
ops since a write to backfill_pos could generate a snap before
backfill_pos, and we assume that (0, backfill_pos) is fully
backfilled. This is a problem since it's possible that
backfill_pos is a valid object, but not one that currently exists.
For example, it might have been deleted since last_backfill was
last changed. Instead, we will explicitly delay ops on
backfill_pos in waiting_for_backfill_pos.
This error resulted in #2691 since wait_for_degraded_object also
attempts to recover the object. At this point, the primary would
attempt to recover the object, find that it isn't there, and put
it in the missing set with need=0,0. Eventually, recover_primary
attempts to recover that object, finds that it has been deleted
in the log, and asserts.
Samuel Just [Tue, 25 Sep 2012 18:07:07 +0000 (11:07 -0700)]
PG: last_complete might be outside of the log
When we encounter a divergent log entry, we put the
object into the missing set at the prior_version
for the divergent event. Unfortunately, the event
at prior_version might have been trimmed leaving
the missing set with an item with a need prior to
log_tail. Thus, last_complete also ends up being
prior to log_tail.
The OSD processes only a single string. The existing example wouldn't
even be passed to the OSD, since it would be interpreted as an option
to the ceph command.
* document osd capabilities
* fix librados user example
* fix example with outdated syntax (pool= and uid= are not supported)
* ignore auid, object prefix, and class restrictions for now since
they aren't usable yet
* fix header for keyring file section
* note that it's only for librbd
* put settings in the [client] section for clarity
* fix typo
* re-indent and clarify sentence about clustered fs on top of RBD
Sam Lang [Wed, 26 Sep 2012 02:11:45 +0000 (19:11 -0700)]
client: Fix coverity 716909 [FORWARD_NULL]
If we fill the request path from the dentry instead of the inode, we
need to use the dentry inode. This fixes a segfault that would occur
in the case where request->inode is in fact null.
Sam Lang [Tue, 25 Sep 2012 16:13:26 +0000 (09:13 -0700)]
client: Fixes coverity defect 716891
Fix for coverity defect 716891 [DEADCODE] in
Client::encode_dentry_release where the released var is initialized
to 0, but never set, so the pre-emptive release of the dentry
capability (if requested by the drop flag) never happens.
Sam Lang [Tue, 25 Sep 2012 16:07:00 +0000 (09:07 -0700)]
client: Fixes coverity defect 716890 [DEADCODE]
In choose_target_mds, the mds number is initialized to 0 and
then a separate target mds number is chosen based on the inode
set in the request. If the inode isn't set, we drop to choosing
a random mds, but only if the mds number is < 0, which doesn't
happen with the initialize to 0. This fix initializes the mds
number to -1, ensuring a random mds is chosen.
Sam Lang [Tue, 25 Sep 2012 14:03:57 +0000 (07:03 -0700)]
cephfs: Fixes for coverity bugs 716840 and 716970
Fixes for two coverity bugs in cephfs.cc.
716840 (CHECKED_RETURN) reported that the return value from
fstat wasn't getting checked. It now is checked, reports an
error and returns to the caller.
716970 (NEGATIVE_RETURNS) reported that the file descriptor passed
to fstat (*fd) might be negative, which isn't allowed. The check at
the top that open failed wasn't aborting the function (when *fd < 0),
now it does.
Mike Ryan [Fri, 14 Sep 2012 17:30:17 +0000 (10:30 -0700)]
timer: add unsafe callbacks option
Using unsafe callbacks drops the lock between invocations of event
callbacks. It is useful under some circumstances, but the user must take
caution. See the comment in Timer.h for full details.
Samuel Just [Tue, 25 Sep 2012 22:47:34 +0000 (15:47 -0700)]
FlatIndex: terminate dir_name buffer passed to strncpy
CID 716836: Buffer not null terminated (BUFFER_SIZE_WARNING)At (1): Calling
strncpy with a maximum size argument of 4096 bytes on destination array
"dir_name" of size 4096 bytes might leave the destination string unterminated.
Samuel Just [Tue, 25 Sep 2012 22:46:44 +0000 (15:46 -0700)]
FileStore: uninit var in ctor OpSequencer::parent
CID 717334: Uninitialized pointer field (UNINIT_CTOR)At (2): Non-static class
member "parent" is not initialized in this constructor nor in any functions
that it calls.
Samuel Just [Tue, 25 Sep 2012 22:18:13 +0000 (15:18 -0700)]
FileStore: fix leaked fd and check lseek in _test_fiemap
CID 716861: Other violation (CHECKED_RETURN)At (3): Calling function "lseek(fd,
off, 0)" without checking return value. This library function may fail and
return an error code.
CID 717090: Resource leak (RESOURCE_LEAK)At (10): Handle variable "fd" going
out of scope leaks the handle.
Samuel Just [Tue, 25 Sep 2012 22:11:25 +0000 (15:11 -0700)]
FileStore: fix statfs,fstatfs,fstat ret val checks
CID 716863: Other violation (CHECKED_RETURN)At (20): Calling function
"fstat(basedir_fd, &basest)" without checking return value. This library
function may fail and return an error code.
Samuel Just [Tue, 25 Sep 2012 22:05:09 +0000 (15:05 -0700)]
FileJournal: uninit var in ctor aio_ctx FileJournal
CID 717333: Uninitialized pointer field (UNINIT_CTOR)At (2): Non-static class
member "aio_ctx" is not initialized in this constructor nor in any functions
that it calls.
Samuel Just [Tue, 25 Sep 2012 21:59:07 +0000 (14:59 -0700)]
FileJournal: uninit var in ctor FileJournal
CID 717332: Uninitialized pointer field (UNINIT_CTOR)At (12): Non-static class
member field "iocb.aio_fildes" is not initialized in this constructor nor in
any functions that it calls.
Samuel Just [Tue, 25 Sep 2012 21:19:49 +0000 (14:19 -0700)]
FileJournal: check return value of lseek in write_fd
CID 716859: Other violation (CHECKED_RETURN)At (1): Calling function
"lseek64(this->fd, pos, 0)" without checking return value. This library
function may fail and return an error code.
Sam Lang [Mon, 24 Sep 2012 16:55:25 +0000 (09:55 -0700)]
client: Fix for #3184 cfuse segv with no keyring
Fixes bug #3184 where the ceph-fuse client segfaults if authx is
enabled but no keyring file is present. This was due to the
client->init() return value not getting checked.
Sage Weil [Sat, 22 Sep 2012 05:20:09 +0000 (22:20 -0700)]
mds: fix ino release on session close
If we project an inode release, we need to follow through, even if for
some reason we decide to keep the session active. Clear out the session
interval_sets in case that happens.
Fixes: #1677 Signed-off-by: Sage Weil <sage@inktank.com>
Sam Lang [Wed, 26 Sep 2012 00:55:05 +0000 (17:55 -0700)]
mds: Handle empty relpath from client getattr
A bug in the client (see 2b54391) results in an empty relpath on
a lookup request. This causes a segfault in the mds, because the
getattr logic expects a lookup to have a relpath to place in the
response. The fix here ensures that lookups include a non-empty
relpath, o.w. aborting the request and returning -EINVAL to the
client.
Samuel Just [Tue, 25 Sep 2012 21:18:19 +0000 (14:18 -0700)]
FileJournal: fix potential overflow in create()
CID 717016: Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)At (1):
Potentially overflowing expression "g_conf->osd_max_write_size << 20" with type
"int" (32 bits, signed) is evaluated using 32-bit arithmetic before being used
in a context which expects an expression of type "int64_t" (64 bits, signed).
To avoid overflow, cast the left operand to "int64_t" before performing the
left shift.
Sam Lang [Tue, 25 Sep 2012 21:48:32 +0000 (14:48 -0700)]
client: Fix to client filepath initializing
The filepath constructor that takes a const char * is missing the
initializer for the encoded member. This results in uninitialized
memory, so the encoded field is sometimes true, resulting in mds
crashes (see #3186) and client errors with empty components in path
names. This commit fixes #3186 and #2285.
Samuel Just [Tue, 25 Sep 2012 21:10:11 +0000 (14:10 -0700)]
osd_types: uninit var in ctor pg_query_t
CID 717346: Uninitialized scalar field (UNINIT_CTOR)At (2): Non-static class
member "epoch_sent" is not initialized in this constructor nor in any functions
that it calls.
Samuel Just [Tue, 25 Sep 2012 20:49:00 +0000 (13:49 -0700)]
PG: uninit var in ctor Scrubber
CID 728001: Uninitialized scalar field (UNINIT_CTOR)At (2): Non-static class
member "epoch_start" is not initialized in this constructor nor in any
functions that it calls.
Samuel Just [Tue, 25 Sep 2012 20:46:32 +0000 (13:46 -0700)]
OSDMap: clear setprecision in print_osd_line
CID 727987: Not restoring ostream format (STREAM_FORMAT_STATE)At (5): Changing
format state of stream "out" for category precision without later restoring it.
mds: Adjust mydir auth when starting MDS that was stopped cleanly
When starting a MDS that was stopped cleanly, we need manually
adjust mydir's auth. This is because MDS log is empty in this case,
mydir's auth can not be adjusted during log replay.
librados: fix use without NULL check in rados_pool_list
CID 716911: Dereference after null check (FORWARD_NULL)
At (5): Passing null pointer "b" to function "strncat(char *, char
const *, size_t)", which dereferences it. (The dereference is assumed
on the basis of the 'nonnull' parameter attribute.)
librados: init everything in default IoCtxImpl ctor
CID 717219: Uninitialized pointer field (UNINIT_CTOR)
At (14): Non-static class member "objecter" is not initialized in this
constructor nor in any functions that it calls.
rbd: make sure we have a device before trying to unmap
CID 717444: Explicit null dereferenced (FORWARD_NULL)
At (48): Passing null pointer "devpath" to function
"do_kernel_rm(char const *)", which dereferences it.
Order is never actually this high currently, but it be via librbd.
CID 716937: Overflowed return value (INTEGER_OVERFLOW)
At (3): Overflowed or truncated value (or a value computed from an
overflowed or truncated value) "offset" used as return value.
CID 717012: Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
At (1): Potentially overflowing expression "1 << obj_order" with type
"int" (32 bits, signed) is evaluated using 32-bit arithmetic before
being used in a context which expects an expression of type "uint64_t"
(64 bits, unsigned). To avoid overflow, cast the left operand to
"uint64_t" before performing the left shift.
CID 717011: Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
At (1): Potentially overflowing expression "1 << order" with type
"int" (32 bits, signed) is evaluated using 32-bit arithmetic before
being used in a context which expects an expression of type "uint64_t"
(64 bits, unsigned). To avoid overflow, cast the left operand to
"uint64_t" before performing the left shift.
CID 717013: Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
At (1): Potentially overflowing expression "1 << order" with type
"int" (32 bits, signed) is evaluated using 32-bit arithmetic before
being used in a context which expects an expression of type "uint64_t"
(64 bits, unsigned). To avoid overflow, cast the left operand to
"uint64_t" before performing the left shift.
CID 717226: Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "cookie" is not initialized in this
constructor nor in any functions that it calls.
librbd: init m_req in LibrbdWriteback::C_Read ctor
CID 717225: Uninitialized pointer field (UNINIT_CTOR)
At (2): Non-static class member "m_req" is not initialized in this
constructor nor in any functions that it calls.
librbd: initialize on-disk header in ImageCtx ctor
CID 717224: Uninitialized scalar field (UNINIT_CTOR)
At (26): Non-static class member field "header.snaps" is not
initialized in this constructor nor in any functions that it calls.
librbd: init everything in default AioRequest constructors
CID 717222: Uninitialized pointer field (UNINIT_CTOR)
At (16): Non-static class member "m_hide_enoent" is not initialized
in this constructor nor in any functions that it calls.
CID 717223: Uninitialized scalar field (UNINIT_CTOR)
At (4): Non-static class member "m_has_parent" is not initialized in
this constructor nor in any functions that it calls.
CID 717220: Uninitialized pointer field (UNINIT_CTOR)
At (4): Non-static class member "aio_type" is not initialized in this
constructor nor in any functions that it calls.
CID 717221: Uninitialized pointer field (UNINIT_CTOR)
At (2): Non-static class member "m_req" is not initialized in this
constructor nor in any functions that it calls.
C_aio_sparse_read_Ack's members are filled in by the only caller
immediately after construction, so make them arguments to the
constructor.
CID 717218: Uninitialized pointer field (UNINIT_CTOR)
At (4): Non-static class member "m" is not initialized in this
constructor nor in any functions that it calls.
WatchContext: these variables are unused, so remove them.
CID 717217: Uninitialized scalar field (UNINIT_CTOR)
At (4): Non-static class member "ver" is not initialized in this
constructor nor in any functions that it calls.
Samuel Just [Fri, 7 Sep 2012 01:02:08 +0000 (18:02 -0700)]
osd/: add PG_STATE_BACKFILLING
PG_STATE_BACKFILLING is set when the pg enters the Backfilling state.
That is, +backfilling indicates that the pg has obtained its
reservations and is now actively backfilling.
Samuel Just [Thu, 6 Sep 2012 22:11:57 +0000 (15:11 -0700)]
osd/: add backfill reservations
Previously, a new osd would be bombarded by backfills from many osds
simultaneously, resulting in excessively high load. Instead, we
want to limit the number of backfills coming into and going out
from a single osd.
To that end, each OSDService now has two AsyncReserver instances: one
for backfills going from the osd (local_reserver) and one for backfills
going to the osd (remote_reserver). For a primary to initiate a
backfill, it must first obtain a reservation from its own
local_reserver. Then, it must obtain a reservation from the backfill
target's remote_reserver via a MBackfillReserve message. This process is
managed by substates of Active and ReplicaActive (see the changes in
PG.h). The reservations are dropped either on the Backfilled event,
which is sent on the primary before calling recovery_complete and on the
replica on receipt of the BackfillComplete progress message), or upon
leaving Active or ReplicaActive.
It's important that we always grab the local reservation before the
remote reservation in order to prevent a circular dependency.
projects / ceph.git / log