init: fix typo s/{$update_crush/${update_crush/

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

Merge branch 'next'

test: use older names for module setup/teardown

setUp and tearDown require nosetests 0.11, but 0.10.4 is the latest on
centos. Rename to use the older aliases, which still work with newer
versions of nosetests as well.

Fixes: #6368
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

doc: Added fixes to osd reporting section.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #854 from ceph/wip-osd-bench-size

osd: fix bench block size

Reviewed-by: Dan Mick <dan.mick@inktank.com>

doc: warn about #6796 in release notes

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

man: re-generate changed man pages

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

RBD Documentation and Example fixes for --image-format

 - RBD Documentation, --image-format wrongly specified as --format in examples
 - RBD Documentation, better describe image format, to differentiate from --format

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

osd: fix bench block size

The command was declared to take 'size' in dumpling, but was trying to
read 'bsize' instead, so it always used the default of 4MiB. Change
the bench command to read 'size', so it matches what existing clients
are sending.

Fixes: #6795
Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'next'

Revert "JounralingObjectStore: journal->committed_thru after replay"

committed_thru can race with the sync thread.

This reverts commit d8d27f13e11dcaefd3aa1c049b97c980283da575.
Reviewed-by: Greg Farnum <greg@inktank.com>

release-notes: clarify that the osd data directory needs to be mounted

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

doc/release-notes.rst: v0.72.1 release notes

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

doc: Fix wrong package names on Debian Wheezy for building Ceph

Merge branch 'master' of https://github.com/ceph/ceph

Merge remote-tracking branch 'upstream/next'

ceph-filestore-tool: add tool for fixing lost objects

Used to repair: #6761
Backport: emperor
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

osd_types: fix object_info_t backwards compatibility

Shipping an object_info_t to a replica with the dirty
flag set would cause the replica to interpret that
object as being lost.  Instead, we always encode
lost into the slot where dumpling expects to find
it and add another field at the end of the encoding.

Backport: emperor
Fixes: #6761
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

CephContext: unregister lockdep after stopping service thread

Fixes: #6769
Fixes: #6674
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

ReplicatedPG: test for missing head before find_object_context

find_object_context doesn't return EAGAIN for a missing head.
I chose not to change that behavior since it might hide bugs
in the future.  All other callers check for missing on head
before calling into find_object_context because we potentially
need head or snapdir to map a snapid onto a clone.

Backport: emperor
Fixes: 6758
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

JounralingObjectStore: journal->committed_thru after replay

It's possible that the osd stopped between when the filestore
op_seq file was updated and when the journal was trimmed.  In
that case, it's possible that on boot the journal might be
full, and yet not be trimmed because commit_start assumes
there is no work to do.  Calling committed_thru on the journal
ensures that the journal matches committed_seq.

Backport: emperor dumpling
Fixes: 6756
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #851 from ceph/wip-6768

Reviewed-by: Greg Farnum <greg@inktank.com>

Use clearer "local monitor storage" in log messages

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

Merge pull request #839 from kri5/master

doc: Remove redundant command for quick start preflight

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #823 from ceph/port/detect-libresolv

autoconf: check for res_nquery explicitly

Merge pull request #789 from ceph/wip-6683

mon: MonmapMonitor: support 'mon getmap [epoch]'

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #836 from ceph/port/unused-headers

trace: remove unused header

doc/release-notes: fix dup

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Updated Emperor reference to 0.72.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added Emperor upgrade.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added dumpling to the sequence.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Remove redundant command for quick start preflight

Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
This patch removes a few commands, which are redundant with following
commands.

Btw, they are also unrelated with the subtitle of the section, which is
"Add the release key".

Merge pull request #834 from ceph/port/stat

client: use platform-specific stat time members

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #833 from ceph/wip-da-SCA-master

Various fixes from SCA

Reviewed-by: Sage Weil <sage@inktank.com>

Merge branch 'next'

trace: remove unused header

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

client: use platform-specific stat time members

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

mon/MDSMonitor.cc: remove some unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_seek_read.c: remove unused variable 'off64_t so'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Client.cc: remove unused variable from Client::CommandHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_http_client.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

remove unused variable from Objecter::RequestStateHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeExample.h: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/osd_types.cc: use !p.tiers.empty() instead of size()

Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeJerasure.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ErasureCodePlugin.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/ObjectStore.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/buffer.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

v0.72

rgw: deny writes to a secondary zone by non-system users

Fixes: #6678
We don't want to allow regular users to write to secondary zones,
otherwise we'd end up with data inconsistencies.

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #830 from ceph/port/gtest-death-tests

test: Only build death tests on platforms that support them

doc/release-notes: note crush update timeout on startup change

Signed-off-by: Sage Weil <sage@inktank.com>

osdmaptool: fix cli tests

From c22c84a88c22688b6044ab37f65a3fe40dfe1983.

Signed-off-by: Sage Weil <sage@inktank.com>

Ceph: Fix memory leak in chain_flistxattr()

Free allocated memory before return.

Signed-off-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Sage Weil <sage@inktank.com>

ReplicatedPG: don't skip missing if sentries is empty on pgls

Formerly, if sentries is empty, we skip missing.  In general,
we need to continue adding items from missing until we get
to next (returned from collection_list_partial) to avoid
missing any objects.

Fixes: #6633
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #827 from ceph/port/bootstrap

Port/bootstrap

Merge pull request #831 from ceph/port/func-name

assert: choose function-var name on non-gnu

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #819 from xinglin/coverity-fixes

Coverity fixes

Reviewed-by; Sage Weil <sage@inktank.com>

assert: choose function-var name on non-gnu

Selects __PRETTY_FUNCTION__ or __func__. Linux assumes GNU, and chooses
__PRETTY_FUNCTION__ if gcc/g++ versions are favorable.

This also includes a fix in ax_c_var_func.m4:

AC_TRY_COMPILE will wrap the test in main{}, and then GCC will complain
about nested functions. Just use the original main{} body.

diff --git a/m4/ax_c_var_func.m4 b/m4/ax_c_var_func.m4
index 0ad7d2b..8b57563 100644
--- a/m4/ax_c_var_func.m4
+++ b/m4/ax_c_var_func.m4
@@ -57,9 +57,9 @@ AC_DEFUN([AX_C_VAR_FUNC],
 [AC_REQUIRE([AC_PROG_CC])
 AC_CACHE_CHECK(whether $CC recognizes __func__, ac_cv_c_var_func,
 AC_TRY_COMPILE(,
-[int main() {
+[
 char *s = __func__;
-}],
+],
 AC_DEFINE(HAVE_FUNC,,
 [Define if the C complier supports __func__]) ac_cv_c_var_func=yes,
 ac_cv_c_var_func=no) )

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

test: Only build death tests on platforms that support them

googletest does not support death tests on FreeBSD.  I've submitted a
patch upstream that trivially enables support, but in the meantime we
can't compile death tests for Ceph on FreeBSD.

https://groups.google.com/forum/#!topic/googletestframework/tjY6UjWgNOw

Signed-off-by: Alan Somers <asomers@gmail.com>

PG: fix operator<<,log_wierdness log bound warning

Split may cause holes such that head != tail and yet
log.empty().

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

PGLog::rewind_divergent_log: log may not contain newhead

Due to split, there may be a hole at newhead.

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osd/ErasureCodePlugin: close library before return on error

close and free resources for library before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: free allocated memory before return NULL

free memory space pointed by row_ids and ind_to_row before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

mon/MDSMonitor: remove unnecessary assignment

epocharg is not initialized, so it does not make any difference
to assign its value to epoch.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: correct one variable name in jerasure_matrix_to_bitmatrix()

When bitmatrix is NULL, this function returns NULL.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

Merge remote-tracking branch 'gh/wip-doc-radosgw'

Revert "doc: radosgw workaround for OpenStack Horizon bug"

This reverts commit c3c962eb7d2fb5ceea829924af08f8a1471f7acf.

Merge pull request #799 from ceph/wip-doc-install

Wip doc install

Reviewed-by: Neil Levine <neil.levine@inktank.com>

Merge pull request #821 from ceph/port/missing-headers

add missing headers

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #822 from ceph/port/rgw

portability patches in rgw

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #824 from dmick/next

osdmaptool: don't put progress on stdout

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #825 from ceph/port/utime

utime: use to_timespec for conversion

Reviewed-by: Sage Weil <sage@inktank.com>

RadosModel: use sharedptr_registry for snaps_in_use

There might be two concurrent rollback ops each of which
adds snap x to snaps_in_use.  Between when the first
completes and the second completes, snap x may be removed
since the first would have removed snap x from snaps_in_use.
Using sharedptr_registry here avoids this by ensuring that
the snap won't be removed from snaps_in_use until all refs
are gone.

This patch also adds size() to sharedptr_registry.

Fixes: #6719
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osdmaptool: don't put progress on stdout

If one requests JSON output, the progress message pollutes the output;
don't do that, send it to stderr instead

Signed-off-by: Dan Mick <dan.mick@inktank.com>

rgw: add compat file for name service macros

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

rgw: avoid sighandler_t in favor of sig_t

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

crush: add mising header for count

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

auth: add missing header for list

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

mon: add missing header for std::find

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

auth: add missing header file for std::replace

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #811 from dmsimard/wip_6710

Be more verbose in radosgw init script issues, return the right exit codes

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

utime: use to_timespec for conversion

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Add a verbose argument and some verbosity

This allows a user to use "-v|--verbose" to get some insight as to
what could be preventing radosgw from starting properly.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

Verify that radosgw started, return appropriate exit code

Changed "/etc/init.d/radosgw status" into a function that now
returns the appropriate exit code.
We now also verify that radosgw actually started properly.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

We should exit 1 if radosgw is not executable

An exit 0 means everything went okay - we should exit 1 if there is a problem in
starting radosgw.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

autoconf: check for res_nquery explicitly

Creates a test that checks explicitly for res_nquery, which can be a
macro in resolv.h. Defines RESOLV_LIBS that contains any libraries that
need to be linked against.

Notes from later fix:

Based on the 2013-09-30 version of wip-port.  On FreeBSD, one must
include netinet/in.h to get the definitions for stuff in resolv.h.
Also, resolv.h's functions are part of libc instead of libresolv.

Signed-off-by: Alan Somers <asomers@gmail.com>
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

autogen.sh: use glibtoolize when available

libtoolize is called glibtoolize on osx.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

autogen: set exit on error

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #816 from ceph/wip-fadvise

wbthrottle: use posix_fadvise if available

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #815 from ceph/wip-static-cast

assert: use feature test for static_cast

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #813 from ceph/wip-blkdev

support blkdev size query on osx/freebsd

Reviewed-by: Sage Weil <sage@inktank.com>

FileStore::_collection_move_rename: handle missing dst dir on replay

In case of a replay, a missing destination directory indicates that
the destination object and directory have been removed by a later
transaction.  Thus, we need to remove the src object and return
0.

Fixes: #6714
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>