mds/MDS.cc: fix dereference null return value

Fix for:

returned_null: Function "SessionMap::get_session(entity_name_t)" returns
  null (checked 12 out of 14 times)

CID 739601 (#1 of 1): Dereference null return value (NULL_RETURNS)
  dereference: Dereferencing a pointer that might be null
  "this->sessionmap.get_session(entity_name_t::CLIENT(client.v))" when
  calling "MDS::send_message_client_counted(Message *, Session *)"

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: fix possible NULL pointer dereference

Assert if 'cur' is NULL.

CID 966616 (#1 of 1): Dereference null return value (NULL_RETURNS)
  dereference: Dereferencing a pointer that might be null "cur" when
  calling "CInode::is_dir()".

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

doc: update mon cluster rescue process for cuttlefish+

Signed-off-by: Sage Weil <sage@inktank.com>

Fix usage for "ceph osd lost"

Will be superceded, but use this commit to backport

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

mon: implement --extract-monmap <filename>

This will make for a simpler process for
  http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#removing-monitors-from-an-unhealthy-cluster

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #306 from Elbandi/wip-fuse-ioctl

ceph-fuse: add ioctl support

Reviewed-by: Sage Weil <sage@inktank.com>

ceph-monstore-tool: implement getmonmap

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-fuse: add ioctl support

Signed-off-by: Andras Elso <elso.andras@gmail.com>

ceph-disk: fix stat errors with new suppress code

Broken by 225fefe5e7c997b365f481b6c4f66312ea28ed61.

Signed-off-by: Sage Weil <sage@inktank.com>

rbd image_read.sh: test different clone object orders

An rbd clone image can be created with an object order that differs
from that of its parent.  This patch adds testing for that in
qa/workunits/rbd/image_read.sh.  By default, clone images will be
created with an object size twice as big as that of its parent.

For simplicity, when a clone's object order differs from its parent
the order will be either one more than or one less than that of its
parent image, meaning its object size is either double or half of
the size of objects used in the parent.

Signed-off-by: Alex Elder <elder@inktank.com>

rbd image_read.sh: add clone-of-clone testing

Add testing to verify that a snapshot of a clone and a clone of
that snapshot both produce the correct results when read.

Signed-off-by: Alex Elder <elder@inktank.com>

rbd image_read.sh: allocate local file at create time

Move the dd command that touches the last byte in a local file
into create_image() where it belongs (out of fill_original()).

Signed-off-by: Alex Elder <elder@inktank.com>

rbd image_read.sh: a few cleanups

The function boolean_toggle() in qa/workunits/rbd/image_read.sh is
defined but never used.  My intentions were good though.  Fix it and
use it for argument parsing.

Change the minimum supported object order so it matches what the
command line interface enforces.

Assign the initial value of TEST_CLONES from the environment if it's
available.

Change defaults to use format 2 and test clones.

Output details about the parameters of the run even if not being
verbose.

Make the order of assignment of argument variables consistent.

And fix a typo unmap_image().

Signed-off-by: Alex Elder <elder@inktank.com>

rbd: prevent import of a dir

Fixes: #2865
Signed-off-by: Sage Weil <sage@inktank.com>

ceph-monstore-tool: be more verbose when replaying a trace

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-monstore-tool: compact command

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-monstore-tool: dump-keys command

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #303 from ceph/wip-librbd-config-create

Reviewed-by: Sage Weil <sage.weil@inktank.com>

Merge branch 'next'

Conflicts:
debian/ceph.prerm

Merge pull request #304 from javacruft/wip-invoke-rc.d

Use invoke-rc.d in maintainer scripts

os/LevelDBStore: fix compression selection

We were always disabling compression.

Fixes: #5131
Reported-by: Sylvain Munaut <s.munaut@whatever-company.com>
Signed-off-by: Sage Weil <sage@inktank.com>

Use invoke-rc.d in maintainer scripts

Upstart configurations and sysv init scripts should be started and stopped
using invoke-rc.d in maintainer scripts.

This ensures the correct behaviour across both Debian (sysv) and Ubuntu (upstart)
and in restricted environments such as schroot where start/stop of services from
maintainer scripts is normally disabled.

Signed-off-by: James Page <james.page@ubuntu.com>

PG: kill some dead IndexedLog methods

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

OSD: kill old split code, it's been dead for a while

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #260 from ceph/wip-suppress

Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

debian: stop sysvinit on ceph.prerm

Signed-off-by: Sage Weil <sage@inktank.com>

mds: better error check on sessionmap load

Would have helped with #5104.

Signed-off-by: Sage Weil <sage@inktank.com>

mon: be a bit more verbose about osd mark down events

Put these in the cluster log; they are interesting.

Signed-off-by: Sage Weil <sage@inktank.com>

Makefile: add -Wformat-security

Clean up the formatting a bit here too.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/next'

Merge pull request #301 from javacruft/wip-upstart-tasks

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #300 from javacruft/wip-format-security

Fix -Werror=format-security errors

ceph df: fix si units for 'global' stats

si_t expects bytes, but it was being given kilobytes.

Signed-off-by: Mike Kelly <pioto@pioto.org>
(cherry picked from commit 0c2b738d8d07994fee4c73dd076ac9364a64bdb2)

Merge pull request #302 from pioto/ceph-df-unit-fix

Reviewed-by: Sage Weil <sage@inktank.com>

ceph df: fix si units for 'global' stats

si_t expects bytes, but it was being given kilobytes.

Signed-off-by: Mike Kelly <pioto@pioto.org>

Remove stop on from upstart tasks

Upstart tasks don't have to concept of 'stop on' as they
are not long running.

Fix -Werror=format-security errors

Fixup a couple of errors when compilng with -Werror=format-security.

libcephfs: add ceph_get_pool_name()

Convert an int pool id to a pool name.  Useful for making sense of the
output from ceph_get_*_layout()'s poolid output argument.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #253 from Elbandi/wip-getlayout

Reviewed-by: Sage Weil <sage@inktank.com>

doc: Added more glossary-compliant terms and indexing.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added another instance term to the glossary.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Minor improvements to Ceph FS landing page.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

libcephfs: get the pool name of a file path

Signed-off-by: Andras Elso <elso.andras@gmail.com>

libcephfs: get replication factor of a file handle/path

Signed-off-by: Andras Elso <elso.andras@gmail.com>

libcephfs: get file handle/path layout info

Signed-off-by: Andras Elso <elso.andras@gmail.com>

libcephfs: get stripe_unit/stripe_count/object_size/pool_id by file handle/path

Signed-off-by: Andras Elso <elso.andras@gmail.com>

libcephfs: fix typos

Signed-off-by: Andras Elso <elso.andras@gmail.com>

Client: get describe_layout by file handle/path

Signed-off-by: Andras Elso <elso.andras@gmail.com>

Merge pull request #295 from ceph/wip-5077

Reviewed-by: Joao Luis <joao.luis@inktank.com>

mon: add 'compact' command

As in, 'ceph mon tell \* compact'

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Luis <joao.luis@inktank.com>

Merge pull request #296 from dalgaaf/wip-da-CID-1021213

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librbd/internal.cc: fix resource leak

Call release() on librados::AioCompletion to free storage before
leave the loop or call new again.

CID 1021213 (#1 of 1): Resource leak (RESOURCE_LEAK)
  leaked_storage: Variable "rados_completion" going out of scope leaks
  the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

doc: Omitted literal syntax from toc.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added fuse syntax to the fstab doc.

fixes: #3672

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'next'

sysvinit: fix enumeration of local daemons when specifying type only

- prepend $local to the $allconf list at the top
- remove $local special case for all case
- fix the type prefix checks to explicitly check for prefixes

Fugly bash, but works!

Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

udev: install disk/by-partuuid rules

Wheezy's udev (175-7.2) has broken rules for the /dev/disk/by-partuuid/
symlinks that ceph-disk relies on.  Install parallel rules that work.  On
new udev, this is harmless; old older udev, this will make life better.

Fixes: #4865
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>

mon: clear pg delta after some period

If we have not pg_map updates, the delta doesn't update, and can get stuck
with the velocity right before activity stopped.  This is confusing, and
can cause incorrect health warnings about in-progress recovery.

To fix this, zero the delta if there is no activity for
'mon delta reset interval' seconds.

Fixes: #5077
Signed-off-by: Sage Weil <sage@inktank.com>

os/FileStore: fix replay guard error msgs (again)

Signed-off-by: Sage Weil <sage@inktank.com>

test_filestore_idempotent_sequence: unmount prior to deleting store

FileStoreDiff umounts the stores in its destructor.

Also, DeterministicOpSequence handles deletes its passed
object store.

Fixes: #5076
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Revert "test_filejournal.cc: cleanup memory in destructor"

The finish() method for Contexts calls delete this.

This reverts commit 36028916c4630ea66007760efed8fc6c441e7af5.

Fixes: #5075
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

librbd: make image creation defaults configurable

Programs using older versions of the image creation functions can't
set newer parameters like image format and fancier striping.

Setting these options lets them use all the new functionality without
being patched and recompiled to use e.g. rbd_create3().
This is particularly useful for things like qemu-img, which does not
know how to create format 2 images yet.

Refs: #5067
backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

os/FileStore: print error code to log on replay guard failure

Signed-off-by: Sage Weil <sage@inktank.com>

rbd.py: fix stripe_unit() and stripe_count()

These matched older versions of the functions, but would segfault
using the current versions.

backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

cls_rbd: make sure stripe_unit is not larger than object size

Test a few other cases too.

backport: cuttlefish, bobtail
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

doc: Added index reference.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added glossary references and index references.

fixes: #3321

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added cluster map and CRUSH definitions.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Fixing index references.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added latency comment.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

debian: make radosgw require matching version of librados2

...indirectly via ceph-common.  We get bad behavior when they diverge, I
think because of libcommon.la being linked both statically and dynamically.

Fixes: #4997
Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

Merge pull request #291 from dalgaaf/wip-da-CID-1019548

client/Client.cc: fix/silence "logically dead code" CID-Error

Even money that this satisfies the coverity gods...

Reviewed-by: Sage Weil <sage@inktank.com>

FileJournal: adjust write_pos prior to unlocking write_lock

In committed_thru, we use write_pos to reset the header.start value in cases
where seq is past the end of our journalq.  It is therefore important that the
journalq be updated atomically with write_pos (that is, under the write_lock).

The call to align_bl() is moved into do_write in order to ensure that write_pos
is adjusted correctly prior to write_bl().

Also, we adjust pos at the end of write_bl() such that pos \in [get_top(),
header.max_size) after write_bl().

Fixes: #5020
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

client/Client.cc: fix/silence "logically dead code" CID-Error

Fix handling of 'safe' and the conditions after calling file_flush().

CID 1019548 (#1 of 1): Logically dead code (DEADCODE)
  dead_error_begin: Execution cannot reach this statement "flock.Unlock();"

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds: avoid assert after suicide()

Fixes: #5079
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #290 from dalgaaf/wip-da-SCA-cppcheck-v2

Reviewed-by: Sage Weil <sage@inktank.com>
rgw bits Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

objclass/class_debug.cc: reduce scope of 'n' in cls_log()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/ceph-filestore-dump.cc: reduce scope of 'r' in export_files()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/system/systest_runnable.cc: reduce scope of 'ret' in join()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/system/st_rados_list_objects.cc: reduce scope of 'ret' in run()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/system/st_rados_create_pool.cc_ reduce scope of 'ret' in run()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/system/rados_list_parallel.cc: reduce scope of 'ret'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librbd/test_librbd.cc: reduce scope of several variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_tools.cc: reduce scope of 'ret' in rgw_get_obj()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_common.cc: reduce scope of 'end' in two cases

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_bucket.cc: reduce scope of 'max' in rgw_remove_bucket()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_admin.cc: reduce scope of 'ret'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_acl_s3.cc: remove local variable 'ret' from create_from_headers()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rbd_fuse/rbd-fuse.c: reduce scope of some variables in open_rbd_image()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Objecter.cc: reduce scope of skipped_map

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/chain_xattr.cc: reduce scope of local variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/os/LFNIndex.cc: reduce scope of suffix_len

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/HashIndex.cc: reduce scope of a local variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: reduce scope of a local variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/os/FlatIndex.cc: reduce scope of suffix_len

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/os/DBObjectMap.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mount/mount.ceph.c: reduce scope of 'skip' in parse_options()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/mds/flock.cc: reduce scope of old_lock_to_end in two cases

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: reduce scope of forced_change_max

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/crush/mapper.c: reduce scope of some local variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>