mon/OSDMonitor: include osd metadata in 'ceph report'

Signed-off-by: Sage Weil <sage@inktank.com>

mon/OSDMonitor: move osd metadata dump into a helper

Signed-off-by: Sage Weil <sage@inktank.com>

mon/OSDMonitor: 'osd metadata N' command

Report recorded metadata about an OSD.

Signed-off-by: Sage Weil <sage@inktank.com>

mon/OSDMonitor: record osd metadata key/value info

Shove what the OSD gives us on boot into the store.

Signed-off-by: Sage Weil <sage@inktank.com>

osd: send host/kernel metadata to mon on boot

Send a bunch of interesting information about the host we are running on
to the monitor on startup.

Signed-off-by: Sage Weil <sage@inktank.com>

doc/release-notes: fix dup

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Updated Emperor reference to 0.72.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added Emperor upgrade.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added dumpling to the sequence.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #834 from ceph/port/stat

client: use platform-specific stat time members

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #833 from ceph/wip-da-SCA-master

Various fixes from SCA

Reviewed-by: Sage Weil <sage@inktank.com>

Merge branch 'next'

client: use platform-specific stat time members

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

mon/MDSMonitor.cc: remove some unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_seek_read.c: remove unused variable 'off64_t so'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Client.cc: remove unused variable from Client::CommandHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_http_client.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

remove unused variable from Objecter::RequestStateHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeExample.h: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/osd_types.cc: use !p.tiers.empty() instead of size()

Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeJerasure.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ErasureCodePlugin.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/ObjectStore.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/buffer.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

v0.72

rgw: deny writes to a secondary zone by non-system users

Fixes: #6678
We don't want to allow regular users to write to secondary zones,
otherwise we'd end up with data inconsistencies.

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #830 from ceph/port/gtest-death-tests

test: Only build death tests on platforms that support them

doc/release-notes: note crush update timeout on startup change

Signed-off-by: Sage Weil <sage@inktank.com>

osdmaptool: fix cli tests

From c22c84a88c22688b6044ab37f65a3fe40dfe1983.

Signed-off-by: Sage Weil <sage@inktank.com>

Ceph: Fix memory leak in chain_flistxattr()

Free allocated memory before return.

Signed-off-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Sage Weil <sage@inktank.com>

ReplicatedPG: don't skip missing if sentries is empty on pgls

Formerly, if sentries is empty, we skip missing.  In general,
we need to continue adding items from missing until we get
to next (returned from collection_list_partial) to avoid
missing any objects.

Fixes: #6633
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #827 from ceph/port/bootstrap

Port/bootstrap

Merge pull request #831 from ceph/port/func-name

assert: choose function-var name on non-gnu

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #819 from xinglin/coverity-fixes

Coverity fixes

Reviewed-by; Sage Weil <sage@inktank.com>

assert: choose function-var name on non-gnu

Selects __PRETTY_FUNCTION__ or __func__. Linux assumes GNU, and chooses
__PRETTY_FUNCTION__ if gcc/g++ versions are favorable.

This also includes a fix in ax_c_var_func.m4:

AC_TRY_COMPILE will wrap the test in main{}, and then GCC will complain
about nested functions. Just use the original main{} body.

diff --git a/m4/ax_c_var_func.m4 b/m4/ax_c_var_func.m4
index 0ad7d2b..8b57563 100644
--- a/m4/ax_c_var_func.m4
+++ b/m4/ax_c_var_func.m4
@@ -57,9 +57,9 @@ AC_DEFUN([AX_C_VAR_FUNC],
 [AC_REQUIRE([AC_PROG_CC])
 AC_CACHE_CHECK(whether $CC recognizes __func__, ac_cv_c_var_func,
 AC_TRY_COMPILE(,
-[int main() {
+[
 char *s = __func__;
-}],
+],
 AC_DEFINE(HAVE_FUNC,,
 [Define if the C complier supports __func__]) ac_cv_c_var_func=yes,
 ac_cv_c_var_func=no) )

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

test: Only build death tests on platforms that support them

googletest does not support death tests on FreeBSD.  I've submitted a
patch upstream that trivially enables support, but in the meantime we
can't compile death tests for Ceph on FreeBSD.

https://groups.google.com/forum/#!topic/googletestframework/tjY6UjWgNOw

Signed-off-by: Alan Somers <asomers@gmail.com>

PG: fix operator<<,log_wierdness log bound warning

Split may cause holes such that head != tail and yet
log.empty().

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

PGLog::rewind_divergent_log: log may not contain newhead

Due to split, there may be a hole at newhead.

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osd/ErasureCodePlugin: close library before return on error

close and free resources for library before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: free allocated memory before return NULL

free memory space pointed by row_ids and ind_to_row before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

mon/MDSMonitor: remove unnecessary assignment

epocharg is not initialized, so it does not make any difference
to assign its value to epoch.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: correct one variable name in jerasure_matrix_to_bitmatrix()

When bitmatrix is NULL, this function returns NULL.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

Merge remote-tracking branch 'gh/wip-doc-radosgw'

Revert "doc: radosgw workaround for OpenStack Horizon bug"

This reverts commit c3c962eb7d2fb5ceea829924af08f8a1471f7acf.

Merge pull request #799 from ceph/wip-doc-install

Wip doc install

Reviewed-by: Neil Levine <neil.levine@inktank.com>

Merge pull request #821 from ceph/port/missing-headers

add missing headers

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #822 from ceph/port/rgw

portability patches in rgw

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #824 from dmick/next

osdmaptool: don't put progress on stdout

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #825 from ceph/port/utime

utime: use to_timespec for conversion

Reviewed-by: Sage Weil <sage@inktank.com>

RadosModel: use sharedptr_registry for snaps_in_use

There might be two concurrent rollback ops each of which
adds snap x to snaps_in_use.  Between when the first
completes and the second completes, snap x may be removed
since the first would have removed snap x from snaps_in_use.
Using sharedptr_registry here avoids this by ensuring that
the snap won't be removed from snaps_in_use until all refs
are gone.

This patch also adds size() to sharedptr_registry.

Fixes: #6719
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osdmaptool: don't put progress on stdout

If one requests JSON output, the progress message pollutes the output;
don't do that, send it to stderr instead

Signed-off-by: Dan Mick <dan.mick@inktank.com>

rgw: add compat file for name service macros

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

rgw: avoid sighandler_t in favor of sig_t

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

crush: add mising header for count

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

auth: add missing header for list

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

mon: add missing header for std::find

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

auth: add missing header file for std::replace

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #811 from dmsimard/wip_6710

Be more verbose in radosgw init script issues, return the right exit codes

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

utime: use to_timespec for conversion

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Add a verbose argument and some verbosity

This allows a user to use "-v|--verbose" to get some insight as to
what could be preventing radosgw from starting properly.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

Verify that radosgw started, return appropriate exit code

Changed "/etc/init.d/radosgw status" into a function that now
returns the appropriate exit code.
We now also verify that radosgw actually started properly.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

We should exit 1 if radosgw is not executable

An exit 0 means everything went okay - we should exit 1 if there is a problem in
starting radosgw.

Signed-off-by: David Moreau Simard <dmsimard@iweb.com>

autogen.sh: use glibtoolize when available

libtoolize is called glibtoolize on osx.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

autogen: set exit on error

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #816 from ceph/wip-fadvise

wbthrottle: use posix_fadvise if available

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #815 from ceph/wip-static-cast

assert: use feature test for static_cast

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #813 from ceph/wip-blkdev

support blkdev size query on osx/freebsd

Reviewed-by: Sage Weil <sage@inktank.com>

FileStore::_collection_move_rename: handle missing dst dir on replay

In case of a replay, a missing destination directory indicates that
the destination object and directory have been removed by a later
transaction.  Thus, we need to remove the src object and return
0.

Fixes: #6714
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #803 from ceph/wip-6698

upstart: fix ceph-crush-location default

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #814 from ceph/wip-da-fix-galois-warning

galois.c: fix compiler warning

Reviewed-by: Loic Dachary <loic@dachary.org>

pdatedoc: Added index link to new install doc. Requires merge of wip-doc-install to work.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Deleted old manual install doc.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Mentioned that install procedure is for manual deployments, not ceph-deploy or others.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Changed text for ceph-extras. Made it required. Mentioned newer versions.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

galois.c: fix compiler warning

galois_create_split_w8_tables() takes no parameter, remove '8' passed
to the function in one case.

osd/ErasureCodePluginJerasure/galois.c: In function 'galois_w32_region_multiply':
osd/ErasureCodePluginJerasure/galois.c:696:5: warning: call to function 'galois_create_split_w8_tables' without a real prototype [-Wunprototyped-calls]
In file included from osd/ErasureCodePluginJerasure/galois.c:53:0:
osd/ErasureCodePluginJerasure/galois.h:71:12: note: 'galois_create_split_w8_tables' was declared here

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

assert: use feature test for static_cast

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

wbthrottle: use posix_fadvise if available

Only adding information about data usage. This won't effect correctness.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

doc: Added DNS and SSL dialog.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

OSD: allow project_pg_history to handle a missing map

If we get a peering message for an old map we don't have, we
can throwit out: the sending OSD will learn about the newer
maps and update itself accordingly, and we don't have the
information to know if the message is valid. This situation
can only happen if the sender was down for a long enough time
to create a map gap and its PGs have not yet advanced from
their boot-up maps to the current ones, so we can rely on it

Fixes: #6712
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

OSD: don't clear peering_wait_for_split in advance_map()

I really don't know why I added this...  Ops can be discarded from the
waiting_for_pg queue if we aren't primary simply because there must have
been an exchange of peering events before subops will be sent within a
particular epoch.  Thus, any events in the waiting_for_pg queue must be
client ops which should only be seen by the primary.  Peering events, on
the other hand, should only be discarded if we are in a new interval,
and that check might as well be performed in the peering wq.

Fixes: #6681
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

ReplicatedPG::recover_backfill: adjust last_backfill to HEAD if snapdir

Otherwise, if last_backfill_started is a snapdir, we will fail to send a
transaction for a client IO creating the head object and removing the
snapdir object.  The result will be that head will eventually be
backfilled, but the snapdir object will erroneously not be removed.

Fixes: #6685
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

test: test helper for get_block_device_size

This is the start of a potential unit test for get_block_device_size. An
actual unit test will probably need to be run as root, and either find a
device, have one specified, or create one (e.g. ramdisk)  in a platform
agnostic way. In the mean time, this tool can be run by hand, or called
for a bash script.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

blkdev: support blkdev size query on osx

Support OSX, add checks for platform specific headers.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

doc: Implemented changes suggested from feedback.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #812 from ceph/revert-struct-init

Revert "fix -Wgnu-designator warnings"

Revert "fix -Wgnu-designator warnings"

Struct initialization using dot notation as in C99, but that happens to
not be valid C++, although apparently in C++11 it is valid. We'll need a
way to keep clang from throwing warnings at this initialization style,
either using some macros to detect the comiler environment, or perhaps
moving struct initialization to a C file rather than C++.

This reverts commit 6efc2b54d5ce85fcb4b66237b051bcbb5072e6a3.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #809 from ceph/wip-pgmap

Reviewed-by: Greg Farnum <greg@inktank.com>

test/libcephfs: free cmount after tests finishes

unmount and release cmount at the end of tests

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
Reviewed-by: Sage Weil <sage@inktank.com>

osd/erasurecode: correct one variable name in jerasure_matrix_to_bitmatrix()

When bitmatrix is NULL, this function returns NULL.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #790 from ceph/wip-warnings

ease clang's anxieties

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #793 from ceph/wip-pipe

pipe: use pipe2 feature test; check fcntl retval

Reviewed-by: Sage Weil <sage@inktank.com>

mon/PGMap: use const ref, not pass-by-value

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #806 from jdurgin/wip-xfstests

Don't run racy xfstest 008

Merge pull request #807 from jdurgin/wip-rbd-map-rw

rbd: omit 'rw' option during map

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #804 from jdurgin/wip-rgw-replica-log-next

rgw: don't turn 404 into 400 for the replicalog api

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>