ceph-object-corpus: revert accidental revert

From 5df3024d7847950672169069b9dc8c8dcb876019

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #861 from dachary/init-ceph

init: fix typo s/{$update_crush/${update_crush/

Merge pull request #832 from ceph/wip-flush-5855-review

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

doc: Clarified description on scrub load threshold.

fixes: #6735

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Fixed instruction.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Added commentary to configure pg defaults. Clarified size commentary.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: PG splitting added to docs.

fixes: #6727

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Took out "future" reference to namespaces.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Clarification of terms.

fixes: #6749

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

init: fix typo s/{$update_crush/${update_crush/

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

Merge branch 'next'

test: use older names for module setup/teardown

setUp and tearDown require nosetests 0.11, but 0.10.4 is the latest on
centos. Rename to use the older aliases, which still work with newer
versions of nosetests as well.

Fixes: #6368
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

doc: Added fixes to osd reporting section.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #854 from ceph/wip-osd-bench-size

osd: fix bench block size

Reviewed-by: Dan Mick <dan.mick@inktank.com>

osd: Backfill peers should not be included in the acting set

Create actingbackfill in choose_acting()
Use first backfill target as previously
Add asserts to catch inappropriate use of actingbackfill
Use is_acting() in proc_replica_info() because this is before actingbackfill set
Remove backfill_targets from stray_set to prevent purge_strays from removing collection
Can't check is_replica() anymore for backfill operations since a backfill isn't
a replica due to acting set change.

fixes: #5855

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd: Simple dout() fix

Signed-off-by: David Zafman <david.zafman@inktank.com>

PG: remove unused Peering::flushed

This has been unused for quite some time!

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: don't requeue waiting_for_active unless flushed and active

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG/PGBackend: block all ops other than Pull prior to active

Previously, it was guarranteed that prior to activation, flushed would
be false on a replica.  Now, there may be a period where flushed is true
due to the flush in Stray completing prior to activation and flushed
being false again.  This is necessary since shortly it won't be possible
to determine from the osdmap whether a stray will be activated in a
particular interval.

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: call start_flush in Active() instead of using do_pending_flush()

This way, we don't block an OpWQ thread while waiting on the filestore.

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG,ReplicatedPG: allow multiple flushes to be in progress

Shortly, we won't know precisely when to flush, so Strays
may need to flush twice.

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG,PG: move duplicate FlushedEvt logic info on_flushed()

Signed-off-by: Samuel Just <sam.just@inktank.com>

doc: warn about #6796 in release notes

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

man: re-generate changed man pages

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

RBD Documentation and Example fixes for --image-format

 - RBD Documentation, --image-format wrongly specified as --format in examples
 - RBD Documentation, better describe image format, to differentiate from --format

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

osd: fix bench block size

The command was declared to take 'size' in dumpling, but was trying to
read 'bsize' instead, so it always used the default of 4MiB. Change
the bench command to read 'size', so it matches what existing clients
are sending.

Fixes: #6795
Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'next'

Revert "JounralingObjectStore: journal->committed_thru after replay"

committed_thru can race with the sync thread.

This reverts commit d8d27f13e11dcaefd3aa1c049b97c980283da575.
Reviewed-by: Greg Farnum <greg@inktank.com>

release-notes: clarify that the osd data directory needs to be mounted

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

doc/release-notes.rst: v0.72.1 release notes

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

doc: Fix wrong package names on Debian Wheezy for building Ceph

Merge branch 'master' of https://github.com/ceph/ceph

Merge remote-tracking branch 'upstream/next'

ceph-filestore-tool: add tool for fixing lost objects

Used to repair: #6761
Backport: emperor
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

osd_types: fix object_info_t backwards compatibility

Shipping an object_info_t to a replica with the dirty
flag set would cause the replica to interpret that
object as being lost.  Instead, we always encode
lost into the slot where dumpling expects to find
it and add another field at the end of the encoding.

Backport: emperor
Fixes: #6761
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

CephContext: unregister lockdep after stopping service thread

Fixes: #6769
Fixes: #6674
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

ReplicatedPG: test for missing head before find_object_context

find_object_context doesn't return EAGAIN for a missing head.
I chose not to change that behavior since it might hide bugs
in the future.  All other callers check for missing on head
before calling into find_object_context because we potentially
need head or snapdir to map a snapid onto a clone.

Backport: emperor
Fixes: 6758
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

JounralingObjectStore: journal->committed_thru after replay

It's possible that the osd stopped between when the filestore
op_seq file was updated and when the journal was trimmed.  In
that case, it's possible that on boot the journal might be
full, and yet not be trimmed because commit_start assumes
there is no work to do.  Calling committed_thru on the journal
ensures that the journal matches committed_seq.

Backport: emperor dumpling
Fixes: 6756
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #851 from ceph/wip-6768

Reviewed-by: Greg Farnum <greg@inktank.com>

Use clearer "local monitor storage" in log messages

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

Merge pull request #839 from kri5/master

doc: Remove redundant command for quick start preflight

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #823 from ceph/port/detect-libresolv

autoconf: check for res_nquery explicitly

Merge pull request #789 from ceph/wip-6683

mon: MonmapMonitor: support 'mon getmap [epoch]'

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #836 from ceph/port/unused-headers

trace: remove unused header

doc/release-notes: fix dup

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Updated Emperor reference to 0.72.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added Emperor upgrade.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added dumpling to the sequence.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Remove redundant command for quick start preflight

Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
This patch removes a few commands, which are redundant with following
commands.

Btw, they are also unrelated with the subtitle of the section, which is
"Add the release key".

Merge pull request #834 from ceph/port/stat

client: use platform-specific stat time members

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #833 from ceph/wip-da-SCA-master

Various fixes from SCA

Reviewed-by: Sage Weil <sage@inktank.com>

Merge branch 'next'

trace: remove unused header

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

client: use platform-specific stat time members

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

mon/MDSMonitor.cc: remove some unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_seek_read.c: remove unused variable 'off64_t so'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Client.cc: remove unused variable from Client::CommandHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_http_client.cc: use static_cast<>() instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

remove unused variable from Objecter::RequestStateHook::call()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeExample.h: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/osd_types.cc: use !p.tiers.empty() instead of size()

Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeJerasure.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ErasureCodePlugin.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/ObjectStore.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/buffer.cc: prefer prefix ++operator for non-primitive types

Prefer prefix ++operator for non-primitive types like iterators for
performance reasons. Prefix ++/-- operators avoid creating a temporary
copy.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

v0.72

rgw: deny writes to a secondary zone by non-system users

Fixes: #6678
We don't want to allow regular users to write to secondary zones,
otherwise we'd end up with data inconsistencies.

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge pull request #830 from ceph/port/gtest-death-tests

test: Only build death tests on platforms that support them

doc/release-notes: note crush update timeout on startup change

Signed-off-by: Sage Weil <sage@inktank.com>

osdmaptool: fix cli tests

From c22c84a88c22688b6044ab37f65a3fe40dfe1983.

Signed-off-by: Sage Weil <sage@inktank.com>

Ceph: Fix memory leak in chain_flistxattr()

Free allocated memory before return.

Signed-off-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Sage Weil <sage@inktank.com>

ReplicatedPG: don't skip missing if sentries is empty on pgls

Formerly, if sentries is empty, we skip missing.  In general,
we need to continue adding items from missing until we get
to next (returned from collection_list_partial) to avoid
missing any objects.

Fixes: #6633
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

Merge pull request #827 from ceph/port/bootstrap

Port/bootstrap

Merge pull request #831 from ceph/port/func-name

assert: choose function-var name on non-gnu

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #819 from xinglin/coverity-fixes

Coverity fixes

Reviewed-by; Sage Weil <sage@inktank.com>

assert: choose function-var name on non-gnu

Selects __PRETTY_FUNCTION__ or __func__. Linux assumes GNU, and chooses
__PRETTY_FUNCTION__ if gcc/g++ versions are favorable.

This also includes a fix in ax_c_var_func.m4:

AC_TRY_COMPILE will wrap the test in main{}, and then GCC will complain
about nested functions. Just use the original main{} body.

diff --git a/m4/ax_c_var_func.m4 b/m4/ax_c_var_func.m4
index 0ad7d2b..8b57563 100644
--- a/m4/ax_c_var_func.m4
+++ b/m4/ax_c_var_func.m4
@@ -57,9 +57,9 @@ AC_DEFUN([AX_C_VAR_FUNC],
 [AC_REQUIRE([AC_PROG_CC])
 AC_CACHE_CHECK(whether $CC recognizes __func__, ac_cv_c_var_func,
 AC_TRY_COMPILE(,
-[int main() {
+[
 char *s = __func__;
-}],
+],
 AC_DEFINE(HAVE_FUNC,,
 [Define if the C complier supports __func__]) ac_cv_c_var_func=yes,
 ac_cv_c_var_func=no) )

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

test: Only build death tests on platforms that support them

googletest does not support death tests on FreeBSD.  I've submitted a
patch upstream that trivially enables support, but in the meantime we
can't compile death tests for Ceph on FreeBSD.

https://groups.google.com/forum/#!topic/googletestframework/tjY6UjWgNOw

Signed-off-by: Alan Somers <asomers@gmail.com>

PG: fix operator<<,log_wierdness log bound warning

Split may cause holes such that head != tail and yet
log.empty().

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

PGLog::rewind_divergent_log: log may not contain newhead

Due to split, there may be a hole at newhead.

Fixes: #6722
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osd/ErasureCodePlugin: close library before return on error

close and free resources for library before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: free allocated memory before return NULL

free memory space pointed by row_ids and ind_to_row before return

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

mon/MDSMonitor: remove unnecessary assignment

epocharg is not initialized, so it does not make any difference
to assign its value to epoch.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

osd/erasurecode: correct one variable name in jerasure_matrix_to_bitmatrix()

When bitmatrix is NULL, this function returns NULL.

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

Merge remote-tracking branch 'gh/wip-doc-radosgw'

Revert "doc: radosgw workaround for OpenStack Horizon bug"

This reverts commit c3c962eb7d2fb5ceea829924af08f8a1471f7acf.

Merge pull request #799 from ceph/wip-doc-install

Wip doc install

Reviewed-by: Neil Levine <neil.levine@inktank.com>

Merge pull request #821 from ceph/port/missing-headers

add missing headers

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #822 from ceph/port/rgw

portability patches in rgw

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #824 from dmick/next

osdmaptool: don't put progress on stdout

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #825 from ceph/port/utime

utime: use to_timespec for conversion

Reviewed-by: Sage Weil <sage@inktank.com>

RadosModel: use sharedptr_registry for snaps_in_use

There might be two concurrent rollback ops each of which
adds snap x to snaps_in_use.  Between when the first
completes and the second completes, snap x may be removed
since the first would have removed snap x from snaps_in_use.
Using sharedptr_registry here avoids this by ensuring that
the snap won't be removed from snaps_in_use until all refs
are gone.

This patch also adds size() to sharedptr_registry.

Fixes: #6719
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

osdmaptool: don't put progress on stdout

If one requests JSON output, the progress message pollutes the output;
don't do that, send it to stderr instead

Signed-off-by: Dan Mick <dan.mick@inktank.com>