client/hadoop/CephFSInterface.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

cls/rgw/cls_rgw.cc: prefer prefix --operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Inode.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SyntheticClient.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

CephxKeyServer.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

auth/Auth.h: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/rest_bench.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/test_filejournal.cc: use %u to format unsigned in sprintf()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/system/st_rados_list_objects.cc: remove not needed statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/osd/RadosModel.h: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/osd/Object.cc: catch exception by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/omap_bench.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_mon_workloadgen.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librados/aio.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/encoding/ceph_dencoder.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

bench/dumb_backend.h: remove never executed delete statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_object_map.cc: use %u to format unsigned in snprintf()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_keyvaluedb_iterators.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_xml.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rest_usage.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rest_s3.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: perform initialization of prefix in initialization list

Perform initialization of IntentLogNameFilter:prefix in initialization list
instead of constructor body.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

testing: fix hadoop-internal-tests

The hadoop-internal-tests workunit needs
to be updated in light of our moving to
using stock Hadoop with our hadoop-cephfs
jars.

Signed-off-by: Joe Buck <jbbuck@gmail.com>

ceph-disk-activate: rely on default/configured keyring path

No reason to override the default or configured value here.

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-disk-activate: don't override default or configured osd journal path

There is no reason not to rely on the default or obey any configured
value here.

Fixes: #4031
Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'next'

Merge branch 'wip-rbd-flatten-cache' into next

Reviewed-by: Sage Weil <sage.weil@inktank.com>

debian: stop ceph-mds before uninstalling ceph-mds

Fixes: #4384
Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/next'

log: drop default 'log max recent' from 100k -> 10k

Use less memory.

Signed-off-by: Sage Weil <sage@inktank.com>

librbd: invalidate cache when flattening

The cache stores which objects don't exist. Flatten bypasses the cache
when doing its copyups, so when it is done the -ENOENT from the cache
is treated as zeroes instead of 'need to read from parent'.

Clients that have the image open need to forgot about the cached
non-existent objects as well. Do this during ictx_refresh, while the
parent_lock is held exclusively so no new reads from the parent can
happen until the updated parent metadata is visible, so no new reads
from the parent will occur.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

ObjectCacher: add a method to clear -ENOENT caching

Clear the exists and complete flags for any objects that have exists
set to false, and force any in-flight reads to retry if they get
-ENOENT instead of generating zeros.

This is useful for getting the cache into a consistent state for rbd
after an image has been flattened, since many objects which previously
did not exist and went up to the parent to retrieve data may now exist
in the child.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

ObjectCacher: keep track of outstanding reads on an object

Reads always use C_ReadFinish as a callback (and they are the only
user of this callback). Keep an xlist of these for each object, so
they can remove themselves as they finish. To prevent racing requests
and with discard removing objects from the cache, clear the xlist in
the object destructor, so if the Object is still valid the set_item
will still be on the list.

Make the ObjectCacher constructor take an Object* instead of the pool
and object id, which are derived from the Object* anyway.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

ceph: fix cli test

Signed-off-by: Sage Weil <sage@inktank.com>

Revert "Use start-stop-daemon --chuid option to setuid to www-data."

This reverts commit a99ed038ecaf170574ee75cd5ddb12f252f9bc26.

On second thought, this will require a bit more care to ensure that all
of the paths radosgw needs to read/write from have the correct permissions
in the packages and so forth.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'wip-4135'

Feature: #4135 - osd: noscrub, nodeepscrub osdmap flags

Reviewed-by: Samuel Just <sam.just@inktank.com>

mon: Paxos: increase trim tolerance from 3 to 30.

This increase only means that we'll keep more versions around before we
trim.  It doesn't change the number of versions we'll keep around after
trimming (that's still as much as 'paxos_max_join_drift', i.e. 10), nor
does it change the criteria used to consider a monitor as having drifted
(same rule applies, 'paxos_max_join_drift').

This change however will enable the leader to put off trimming for a longer
period of time, giving a better chance for a monitor to join the cluster.
See, after going through the probing phase, at which point a monitor may
only be, say, 5 versions off, the same monitor may end up getting into the
quorum only to find that in-between probing and finally triggering an
election some 6 versions might have come to existence.  Before this patch,
by then the state had been trimmed and the monitor would have to bootstrap
to perform a full store sync.  With this patch in place, the monitor would
be able to sync the remaining 11 versions.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

mon: Paxos: bootstrap leader if he has fallen behind upon reaching collect

Fixes: #4256
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Document scrubbing behavior

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge pull request #100 from jaharkes/init

Fixes for RadosGW init script

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #99 from dalgaaf/wip-da-sca-style-3

More smaller (style related) fixes

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #91 from Youscribe/defaultweight

Set weight of an osd based on df.

Reviewed-by: Sage Weil <sage@inktank.com>

Use start-stop-daemon --chuid option to setuid to www-data.

Signed-off-by: Jan Harkes <jaharkes@cs.cmu.edu>

Fix radosgw actually reloading after rotating logs.

The --signal argument to Debian's start-stop-daemon doesn't
make it send a signal, but defines which signal should be send
when --stop is specified.

Signed-off-by: Jan Harkes <jaharkes@cs.cmu.edu>

rgw/rgw_op.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_multi_del.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_multi.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_http_client.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados_sync.cc: fix typo in error msg

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_common.cc: initialize has_acl_header in constructor with 'false'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_admin.cc: remove some unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_admin.cc: reduce scope of 'int ret'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_admin.cc: remove twice check for 'object'

Remove twice checked 'object'. Put checks in alphabetic order to make
double checks easier to catch.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.h: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_acl_s3.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_common.h: pass some const variables by reference to 'operator<<'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rbd_fuse/rbd-fuse.c: remove consecutive return/break statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rbd.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados_sync.{cc,h}: pass some const std::string by reference

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados_sync.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados_export.cc: reduce scope of 'int ret' in do_rados_export()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/rados.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/osdmaptool.cc: remove some unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Objecter.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.cc: set left to 0 instead off 'left -= left'

Instead of call 'left -= left', which results in 0, set left directly
to 0 and remove not needed assert call 'left == 0'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: move variable async_done to the right #ifdef block

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: remove assignment of a never used value

There is no need to set local variable basedir_fd to -1 after
closing and right before leaving the function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/SimpleMessenger.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Monitor.cc: Clarify calculation precedence for '%' and '?'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/internal.cc: clarify expression with parentheses

From cppcheck:
 [src/librbd/internal.cc:379]: (style) Boolean result is used in
 bitwise operation. Clarify expression with parentheses.

SyntheticClient.cc: Clarify calculation precedence for '%' and '?'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

fusetrace_ll.cc: reduce scope of 'int err' in main()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: fix warning about 'Possible null pointer dereference'

Fix following warning from cppcheck:
 [src/mds/Locker.cc:2255] -> [src/mds/Locker.cc:2258]: (error)
   Possible null pointer dereference: in - otherwise it is redundant
   to check it against null.

Since head_in used for call pick_inode_snap() is already valid,
there is no need to check if 'CInode *in' is not NULL. Remove
not needed check.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonMap: don't crash on dup IP in mon host

If the mon_host line has an IP twice, we shouldn't crash.

Signed-off-by: Sage Weil <sage@inktank.com>

Add code to check flag state

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge remote-tracking branch 'gh/next'

ceph_common.sh: add warning if 'host' contains dots

This is a common error and there's no reason the script can't
at least tell you it's a really bad idea.  One might argue it
could even successfully proactively truncate the host parameter
at the first dot, but that's a little controlling, perhaps.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Revert "Update conf.py"

This reverts commit 700899206224436255522d8469612e6603fe3116.
Apparently github's edit lets you commit directly to the repo.
Oops.

Update conf.py

osd: noscrub, nodeepscrub osdmap flags

Just add NOSCRUB/NODEEP_SCRUB to OSDMap flags and  allow set/unset

Feature: #4135

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge pull request #96 from concubidated/master

Fix output of command 'ceph osd tree --format=json'

Reviewed-by: Sage Weil <sage@inktank.com>

doc: Added a new network configuration reference document.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Fix output of 'ceph osd tree --format=json'

Signed-off-by: Tyler Brekke <tyler.brekke@inktank.com>

doc: Added network settings. Reorganized index.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Excised network settings.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Format edits. Excised much of network discussion. Added reference to network guide.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'wip-osd-map' into next

Fixes: 4369
Backport: bobtail
Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: mark down connections from old peers

Close out any connection with an old peer.  This avoids a race like:

- peer marked down
- we get map, mark down the con
- they reconnect and try to send us some stuff
- we share our map to tell them they are old and dead, but leave the con
  open
...
- peer marks itself up a few times, eventually reuses the same port
- sends messages on their fresh con
- we discard because of our old con

This could cause a tight reconnect loop, but it is better than wrong
behavior.

Other possible fixes:
 - make addr nonce truly unique (augment pid in nonce)
 - make a smarter 'disposable' msgr state (bleh)

Signed-off-by: Sage Weil <sage@inktank.com>

osd/PG: rename require_same_or_newer_map -> is_same_or_newer_map

This avoids confusion with the OSD method of the same name, and better
matches what the function tests (and does not do).

Signed-off-by: Sage Weil <sage@inktank.com>

mon/Paxos: make old clock skew message more/less suggestive

There is a separate clock skew check now, and more often than not this
suggests laggy messages, and not necessarily skew.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #92 from jaharkes/content_length

Properly format Content-Length: header on 32-bit systems.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Properly format Content-Length: header on 32-bit systems.

- Promote len argument in dump_content_length to uint64_t.
- Make sure there is sufficient scratch space to format string.
- Use PRIu64 macro for formatting.

Signed-off-by: Jan Harkes <jaharkes@cs.cmu.edu>

rgw: set attrs on various list bucket xml results (swift)

Fixes: #4247
The list buckets operation was missing some attrs on the different
xml result entities. This fixes it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>