test/kv_store_bench.cc: fix coverity REVERSE_INULL

CID 1021214 (#1 of 1): Dereference before null check (REVERSE_INULL)
 check_after_deref: Null-checking cb_args suggests that it may be
 null, but it has already been dereferenced on all paths leading to
 the check.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/libradosstriper/striping.cc: remove dead initialization

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Beacon.*: fix UNINIT_CTOR cases

CID 1238905 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member want_state is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member last_send is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeLrc.h: fix UNINIT_CTOR

Fix Coverity issue, preinit with 0:

uninit_member: Non-static class member chunk_count is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeLrc.h: fix data_chunk_count UNINIT_CTOR

CID 1238903 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member data_chunk_count is not
 initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

LogEntry.h: init LogEntry::seq with 0

CID 1238904 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value e.seq when calling
 log_to_syslog.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librados/aio.cc: close resource leak

Release my_completion4.

CID 1219581 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable my_completion4 going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2536 from dachary/wip-doc-reweight

documentation: explain ceph osd reweight vs crush weight

Reviewed-by: Laurent Guerby <laurent@guerby.net>

documentation: explain ceph osd reweight vs crush weight

Using the wording from Gregory Farnum at
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-June/040961.html

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2514 from ceph/wip-acl-test

qa/workunits/fs/misc: Add a workunit for ACL

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2457 from ceph/wip-7767-b

msg: prefetch data when doing recv()

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktnak.com>

Merge pull request #2440 from somnathr/wip-optracker-optimization

Wip optracker optimization

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2445 from wonzhq/req-pending

Cache tiering: do not allow flush/evict when there are requests pending

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2515 from ceph/wip-9518

mds: fix not journaling client metadata

Reviewed-by: Yan, Zheng <ukernel@gmail.com>

mds: fix not journaling client metadata

Previously the code was there for storing in
the SessionMap table, but not for the ESession
logevent.

Fixes: #9518
Signed-off-by: John Spray <john.spray@redhat.com>

qa/workunits/fs/misc: Add a workunit for ACL

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #2503 from ceph/wip-xattr-fix

Wip xattr fix

Merge remote-tracking branch 'origin/giant'

Merge pull request #2498 from ceph/wip-objecter-cancel

osdc/Objecter: cancel timeout before clearing op->session

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #2513 from ceph/wip-decode-dump

msg: dump messages we can' decode at a low devel level

Reviewed-by: Greg Farnum <greg@inktank.com>

msg: hexdump bad messages at debug level 1 (not 30)

These are rare, and when they do happen, we want to see them in the
logs.

Signed-off-by: Sage Weil <sage@redhat.com>

msg: hexdump messages we can't decode at configurable debug level

We had this hard-coded to 30.  Make it tunable.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2512 from dachary/wip-9490-crushtool-num-rep

Revert "crushtool: safeguard for missing --num-rep when --test"

Reviewed-by: John Spray <john.spray@redhat.com>

Revert "crushtool: safeguard for missing --num-rep when --test"

This reverts commit 10b8966c8da38f8ae5ac7dbb2e16a5e45e502ba3.

Merge pull request #2496 from ceph/wip-9189

#9375 and #9189 - add client metadata and improve health messages

Reviewed-by: Greg Farnum greg@inktank.com

mds: use new Session::human_name in health msgs

Signed-off-by: John Spray <john.spray@redhat.com>

mds: implement const SessionMap::get_session

Signed-off-by: John Spray <john.spray@redhat.com>

mds: generate friendly name for client sessions

...from the client metadata, for use in health messages.

Signed-off-by: John Spray <john.spray@redhat.com>

ceph_fuse: include mount point in client metadata

Signed-off-by: John Spray <john.spray@redhat.com>

client: send metadata in session open

Populated with hostname and entity_id by
default, with interface for outer layers
like ceph_fuse, libcephfs to inject their
own metadata.

Signed-off-by: John Spray <john.spray@redhat.com>

ceph_fuse: refactor init to make mount_point accessible

...so that we can read out the mount point between init
and start() in order to feed it to Client before it connects
to the MDS.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: include client metadata in `session ls`

Signed-off-by: John Spray <john.spray@redhat.com>

mds: store client metadata on Session

Signed-off-by: John Spray <john.spray@redhat.com>

messages: add cephfs client metadata to MClientSession

Signed-off-by: John Spray <john.spray@redhat.com>

mds: collapse numerous client warnings into one

...to avoid sending O(N_clients) sized beacons.

Fixes: #9375
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #2508 from dachary/wip-9490-crushtool-num-rep

crushtool: safeguard for missing --num-rep when --test

Reviewed-by: Johnu George <johnugeorge109@gmail.com>

client: request xattrs if xattr_version is 0

Following sequence of events can happen.
 - Client releases an inode, queues cap release message.
 - A 'lookup' reply brings the same inode back, but the reply
   doesn't contain xattrs because MDS didn't receive the cap release
   message and thought client already has up-to-data xattrs.

The fix is force sending a getattr request to MDS if xattrs_version is 0.
The getattr mask is set to CEPH_STAT_CAP_XATTR, so MDS knows client does
not have xattr.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: return xattrs when client explictly requests xattrs

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #2507 from ceph/wip-9435

#9435 - enforce rules for using EC/tiered pools with CephFS

Reviewed-by: Greg Farnum <greg@inktank.com>

mon: forbid tier changes when in use by FS

* Removing tiers from a base pool in use by CephFS is forbidden.
* Using CephFS pools as tiers is forbidden.

Signed-off-by: John Spray <john.spray@redhat.com>

mon: prevent cache pools being used CephFS

Fixes two things:
 * EC pools are now permissible if they have a cache overlay
 * Pools are not permissible if they are a cache tier.

Fixes: #9435
Signed-off-by: John Spray <john.spray@redhat.com>

crushtool: safeguard for missing --num-rep when --test

http://tracker.ceph.com/issues/9490 Fixes: #9490

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2475 from ceph/wip-lttng

Update rbd-replay

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #2473 from ceph/wip-doc-preflight

doc: Revised the Preflight Doc

Reviewed-by: John Wilkins <jowilkin@redhat.com>

doc: Fixed broken hyperlink.

Signed-off-by: John Wilkins <jowilki@redhat.com>

script/run-coverity: update submodules

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2394 from ceph/wip-da-fix-make_check

Fix some issues with make check

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2504 from dachary/wip-erasure-code-valgrind

test: libtool --mode=execute valgrind

Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Merge pull request #2497 from ceph/wip-xfs-inode64

ceph-disk: mount xfs with inode64 by default

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/mon/mkfs.sh: add check for default keyring

The auth_cephx_key check always fails if there is a default
keyring in /etc/ceph/ available. Check if keyring exists and
fail with error message if keyring file in path.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/ceph-disk.sh: resolve symlinks before check

Make sure symlinks are resolved in command_fixture()
before compare result of which command and the current
path.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2487 from wonzhq/org

mailmap: correcting Zhiqiang Wang's mailmap and org affiliation

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

mailmap: correcting Zhiqiang Wang's mailmap and org affiliation

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>

test: libtool --mode=execute valgrind

otherwise valgrind will not actually run against the binary.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

mds: set new inode's xattr version to 1

set new inode's xattr version to 1 even if it has no xattr. This allow
client to differentiate no xattr in inode from MDS skips sending xattr
to client (because MDS think client already has uptodate xattr).

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #2499 from ceph/wip-9219-giant

wip-9219: subscribe to the newest osdmap when reconnecting to a monitor

Reviewed-by: Sage Weil <sage@redhat.com>

osd: subscribe to the newest osdmap when reconnecting to a monitor

This is mostly relevant in testing clusters, but it ensures that an OSD
disconnecting from the monitor at the wrong time will still see any recent
map updates and prevent accidental loss of map injection into the OSD cluster.
Fixes: #9219
Signed-off-by: Greg Farnum <greg@inktank.com>

osdc/Objecter: fix command op cancellation race

Cancel the command op timeout event before we clear out the op from the
session struct.  This isn't strictly necessary because command_op_cancel
will "gracefully" handle the case where the tid is no longer present, but
this avoids that noise and is cleaner.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: cancel timeout before clearing op->session

The C_CancelOp path assumes op->session != NULL.  Cancel that op before
we clear it.  This fixes a crash like

#0  pthread_rwlock_wrlock () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_wrlock.S:39
#1  0x00007fc82690a4b1 in RWLock::get_write (this=0x18, lockdep=<optimized out>) at ./common/RWLock.h:88
#2  0x00007fc8268f4d79 in Objecter::op_cancel (this=0x1f61830, s=0x0, tid=0, r=-110) at osdc/Objecter.cc:1850
#3  0x00007fc8268ba449 in Context::complete (this=0x1f68c20, r=<optimized out>) at ./include/Context.h:64
#4  0x00007fc8269769aa in RWTimer::timer_thread (this=0x1f61950) at common/Timer.cc:268
#5  0x00007fc82697a85d in RWTimerThread::entry (this=<optimized out>) at common/Timer.cc:200
#6  0x00007fc82651ce9a in start_thread (arg=0x7fc7e3fff700) at pthread_create.c:308

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-disk: mount xfs with inode64 by default

We did this forever ago with mkcephfs, but ceph-disk didn't.  Note that for
modern XFS this option is obsolete, but for older kernels it was not the
default.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2492 from ceph/wip-9284

#9284 - fix client RECALL handling and add health metrics

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2482 from dachary/wip-erasure-code-documentation

erasure code documentation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2476 from ceph/wip-9307

rgw: push hash calculater deeper

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2493 from ceph/wip-rbd-objectcacher-hang

rbd: ObjectCacher reads can hang when reading sparse files

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #2495 from dachary/wip-erasure-code-preload

erasure-code: preload fails if < 0

Reviewed-by: Sage Weil <sage@redhat.com>

erasure-code: preload fails if < 0

And not if < -1.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

doc: Added feedback.

Signed-off-by: John Wilkins <jowilki@redhat.com>

Merge pull request #2486 from jgalvez/master

init-radosgw.sysv: Support systemd for starting the gateway

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2472 from dachary/wip-9429-bench

erasure-code: fix erasure_code_benchmark goop (decode)

Reviewed-by: Janne Grunau <j@jannau.net>

mds: limit number of caps inspected in caps_tick

This is to avoid hitting an O(caps) loop in the worst
cast scenario.  This mechanism is a little crude but
should be superceded at some point by admin socket
functionality to inspect session caps so that we
don't need to spit out this level of detail in logs.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: keep per-client revoking caps list

...to avoid doing an O(caps) scan to find out
which clients are responsible for any late-revoking
caps during health checks.

Signed-off-by: John Spray <john.spray@redhat.com>

xlist: implement copy constructor

...so that I can have a std::map of them.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: health metric for late releasing caps

Follow up on Yan Zheng's "mds: warn clients which
aren't revoking cap" to include a health metric
for this condition as well as the clog messages.

Signed-off-by: John Spray <john.spray@redhat.com>

mon: trigger transaction on MDS health changes

I think this was previously only working as a side effect
of other MDS map changes.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add a health metric for failure to recall caps

Fixes: #9284
Signed-off-by: John Spray <john.spray@redhat.com>

mds: add state for tracking RECALL progress

To be used later for generating health metrics
for clients which are failing to promptly service
CEPH_SESSION_RECALL_STATE messages.

Signed-off-by: John Spray <john.spray@redhat.com>

xlist: implement const_iterator

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix trim_caps for inodes in root

Previously client would fail to release caps for files
in the root directory in response to CEPH_SESSION_RECALL_STATE
messages.

Signed-off-by: John Spray <john.spray@redhat.com>

client: failure injection for cap release

Used for simulating a buggy client that trips
the error detection in #9282 (warn clients
which aren't revoking caps)

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix potentially invalid read in trim_caps

trim_dentry can potentially free an inode, so get/put
it around the block where we use the inode's dn_set.

Signed-off-by: John Spray <john.spray@redhat.com>

client: more precise cap trimming

Two fixes:
 * Client would unlink everything it could, instead of just
   meeting its goal, because caps.size() doesn't change until
   dentries are cleaned up later.  Take account of the trimmed
   count in the while() condition to fix that.
 * Don't count the root ino as trimmed, as although it has no
   dentries (of course), we will never give up the cap.

With this change, the client will now precisely achieve the number
of caps requested in CEPH_SESSION_RECALL_STATE messages.

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix crash in trim_caps

In a75af4c2, procedure was added to invalidate root's dentries
if the trimming failed to free enough caps.  This would sometimes
crash because root->dir wasn't necessarily open.

Fix by only doing it if root dir is open, though I suspect this
may not be the end of it...

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #2485 from Abioy/master

bugfix: wrong socket address in log msg of Pipe.cc

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

bugfix: wrong socket address in log msg of Pipe.cc

paddr was not yet set up for the socket address

Signed-off-by: Yongyue Sun abioy.sun@gmail.com

Merge pull request #2442 from dachary/wip-6754-jerasure-parameters

erasure-code: fix BlaumRoth sanity check on w

Reviewed-by: Andreas Peters <andreas.joachim.peters@cern.ch>

Merge pull request #2488 from cernceph/docfix

doc: osd_backfill_scan_(min|max) are object counts

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

doc: osd_backfill_scan_(min|max) are object counts

osd_backfill_scan_min and osd_backfill_scan_max set the number of
items grabbed during a single backfill scan, not an interval in
seconds. Correct the doc.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

rbd: ObjectCacher reads can hang when reading sparse files

The pending read list was not properly flushed when empty objects
were read from a space file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

init-radosgw.sysv: Support systemd for starting the gateway

When using RHEL7 the radosgw daemon needs to start under systemd.

Check for systemd running on PID 1. If it is then start
the daemon using: systemd-run -r <cmd>. pidof returns null
as it is executed too quickly, adding one second of sleep and
script reports startup correctly.

Signed-off-by: JuanJose 'JJ' Galvez <jgalvez@redhat.com>

Merge pull request #2484 from sjahl/master

doc: Added bucket management commands to ops/crush-map

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

doc: Added bucket management commands to ops/crush-map

Describes the CLI for adding and removing buckets, in addition to the
'moving' instructions which were already present.

Signed-off-by: Stephen Jahl <stephenjahl@gmail.com>

Merge remote-tracking branch 'gh/giant'

documentation: jerasure plugin is sub section of erasure code profile

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

documentation: isa plugin is sub section of erasure code profile

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

documentation: erasure code profile update

The plugins are made a sub-section of the erasure code profile
section. The k and m parameters are removed from erasure code profile
documentation. They were added for simplicity in the first version
because there was only one plugin. But they should really be in the
plugin documentation.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2481 from sjahl/master

doc: fixes a formatting error on ops/crush-map

doc: fixes a formatting error on ops/crush-map

Signed-off-by: Stephen Jahl <stephenjahl@gmail.com>

Merge pull request #2467 from majianpeng/fix3

buffer: In rebuild_page_aligned for the last ptr is page aligned, no need call rebuild().

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2478 from ceph/wip-9445

global: fix hang when segv happens inside logging code

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2477 from ceph/wip-client-msg-leak

client: fix a message leak