Merge pull request #3094 from yuandong1222/bufferlist-get_contiguous

common: bufferlist::get_contiguous return 0 when param len == 0

Reviewed-by: Sage Weil <sage@redhat.com>

common: bufferlist::get_contiguous return 0 when param len == 0

Change-Id: I3eaee45b12c634687cd1b294b3c264aed8cab03a
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>

Merge pull request #3081 from dachary/wip-10246-centos7

doc: add centos 7 to ceph-deploy installation

Reviewed by: John Wilkins <jowilkin@redhat.com>

Merge pull request #3031 from dachary/wip-10199-config-get-xml

common: admin sock output XML elements whitespace

Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Merge pull request #2959 from ceph/wip-10104

rados.py: fix misnamed 'wait_*' routines, add true wait/wait-with-cb

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2985 from CzBiX/quick-fix

rgw: check endpoints is not empty before use it.

This fixes a regression due to commit 3c4b782bcb5460523982a04b1e5c7963e08a1687

Reported-by: Mykola Golub <to.my.trociny@gmail.com>

pybind/test_rados: add test for wait_for_complete_and_cb()

Also, factor out some common code between the two tests

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rados.py: fix misnamed 'wait_*' routines, add true wait/wait-with-cb

Also modify aio_read test for wait: write an object, take its active set
down, try to aio_read; verify read doesn't complete until active set is
allowed back up

Fixes: #10104
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #3076 from ceph/wip-mon-mdsdne

MDSMonitor.cc: fix assertion caused by MDSMap::STATE_DNE state

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3072 from ceph/wip-10231-gperftools-location

heap_profiler: support new gperftools header locations

gitbuilders are happy

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3084 from ceph/wip-rados-include

librados: Fix rados include problem reported by "sponce"

Reviewed-by: Sage Weil <sage@redhat.com>

librados: Fix rados include problem reported by "sponce"

Caused by a03f85a8e7fab296ea2df70a929a1c5e4aa0f7fb

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #3059 from dachary/wip-10125-radosgw-init

rgw: run radosgw as apache with systemd

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: run radosgw as apache with systemd

Same as sysv.

http://tracker.ceph.com/issues/10125 Fixes: #10125

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #2997 from ceph/wip-snapmapper

osd: require SNAPMAPPER feature; drop tolerate_collection_add_enoent

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2979 from ceph/wip-kill-category

osd: kill 'category' from pg_stats_t

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2694 from dachary/wip-9728-jerasure-neon

erasure-code: jerasure support for NEON

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3082 from scuttlemonkey/wip-doc-10247

Changed os-recommendation table sorting to remove implied bias.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3048 from ceph/wip-10209

osd: tolerate sessionless con in fast dispatch path

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3049 from dachary/wip-10211-erasure-code-buffer-alignement

erasure-code: enforce chunk size alignment

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/release-notes: v0.89

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/next'

Changed os-recommendation table sorting to remove implied bias.

Fixes: #10247
signed-of by: Patrick McGarry <pmcgarry@redhat.com>

documentation: add centos 7 to ceph-deploy installation

http://tracker.ceph.com/issues/10246 Fixes: #10246

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3024 from ceph/wip-librados-symbols

librados: Only public API symbols from the shared library

Reviewed-by: Loic Dachary <ldachary@redhat.com>

MDSMonitor.cc: fix assertion caused by MDSMap::STATE_DNE state

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3028 from majianpeng/fix4

Add perf reset command.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3074 from ceph/wip-10229

osdc/Filer: use finisher to execute C_Probe and C_PurgeRange

osdc/Filer: use finisher to execute C_Probe and C_PurgeRange

Currently contexts C_Probe/C_PurgeRange are executed while holding
OSDSession::completion_lock. C_Probe and C_PurgeRange may call
Objecter::stat() and Objecter::remove() respectively, which acquire
Objecter::rwlock. This can cause deadlock because there is intermediate
dependency between Objecter::rwlock and OSDSession::completion_lock:

 Objecter::rwlock -> OSDSession::lock -> OSDSession::completion_lock

The fix is exexcute C_Probe/C_PurgeRange in finisher thread.

Fixes: #10229
Signed-off-by: Yan, Zheng <zyan@redhat.com>

heap_profiler: support new gperftools header locations

The google/ headers location has been deprecated as of gperftools 2.0.
As of gperftools 2.2rc, the google/ headers will now give deprecation
warnings, and they will probably disappear in a future gperftools
update.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

mon: fix signed/unsigned comparison warning

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

0.89

Merge pull request #3065 from ceph/wip-cephtool-exec

qa: fix cephtool tests not getting run

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa: fix teardown in cephtool's test_mon_mds

This was missing from 17b5fc9a but we didn't notice
because the test wasn't being run by the gitbuilders.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3044 from osynge/wip=suse-systemd

better systemd support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3067 from dachary/wip-cephtool

fix cephtools running only part of the tests + avoid killing wrong daemons

Reviewed-by: Sage Weil <sage@redhat.com>

common/perf_counters: Remove the duplicated code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test/perf_counters: Replace perfcounters_dump to perf dump.

The func of command  perfcounters_dump and 'perf dump' are the same .
But from the print 'ceph --admin-daemon help', it only print 'perf
dump'. So replace.
In order to keep consistent, still keep perfcounters_dump in code for
old user.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test/perf_counters: Add test case for cmd 'perf reset'

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/perf_counters: Add 'perf reset all|perfcountername' for 'ceph
--admin-daemon'

Using this command, we can reset all perfcounters or reset on specified
perfcounter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #3068 from ktdreyer/doc-os-recommendations-cleanup

doc: os recommendations cleanup

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: rm Fedora minor versions in os recommendations

Fedora doesn't have point releases, so it doesn't make sense to talk
about "Fedora 18.0" or "Fedora 19.0" - it's just "Fedora 18" or "Fedora
19".

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

os/ObjectStore: drop tolerate_collection_add_enoent

This is no longer needed and clutters the interface.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: require SNAPMAPPER feature from peers

This was introduced before cuttlefish.  We require users to upgrade first
to a newer release, so there is no need to support a mixed cluster with
such old code.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3061 from ktdreyer/rpm-ceph-disk-man-page

packaging: package ceph-disk(8)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

packaging: package ceph-disk(8)

The ceph-disk man page was added in
a450cab2b8148cb8a9b043d629feccf89e5aabac, but this was not added to the
RPM or DEB packaging. Add it here.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: rm old releases in os recommendations page

Similar to the change in commit
4cdc9c793627eec13c65543c09c604ec27a4dd8a, remove the references to old
Ceph releases.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc: clarify "B" flag in os recommendations page

We don't exactly do continuous builds on all the platforms marked with
"B", but we have published binary RPMs for them. Adjust the "B"
footnote definition to reflect this.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

qa: cephtool/test.sh use regular strings to list tests

Using the array notation to list test is error prone and more
complicated to write.

It also fixes a bug : only the first test of each series (MON, OSD, MDS)
was run and the others were ignored.

Signed-off-by: Loic Dachary <loic@dachary.org>

tests: vstart_wrapper.sh must call vstart.sh with a list of daemons

The list of daemons is provided in the $CEPH_START environment variable
but vstart_wrapper.sh incorrectly uses the $VSTART_ARGS empty variable.

As a consequence vstart.sh calls init-ceph stop which kills daemons that
do not belong to the test.

http://tracker.ceph.com/issues/10215 Fixes: #10215

Signed-off-by: Loic Dachary <loic@dachary.org>

erasure-code: enforce chunk size alignment

Let say the ErasureCode::encode function is given a 4096 bytes
bufferlist made of a 1249 bytes bufferptr followed by a 2847 bytes
bufferptr, both properly starting on SIMD_ALIGN address. As a result the
second 2048 had to be reallocated when bufferlist::substr_of gets the
second 2048 buffer, the address starts at 799 bytes after the beginning
of the 2847 buffer ptr and is not SIMD_ALIGN'ed.

The ErasureCode::encode must enforce a size alignment based on the chunk
size in addition to the memory alignment required by SIMD operations,
using the bufferlist::rebuild_aligned_size_and_memory function instead of
bufferlist::rebuild_aligned.

http://tracker.ceph.com/issues/10211 Fixes: #10211

Signed-off-by: Loic Dachary <loic@dachary.org>

common: allow size alignment that is not a power of two

Do not assume the alignment is a power of two in the is_n_align_sized()
predicate. When used in the context of erasure code it is common
for chunks to not be powers of two.

Signed-off-by: Loic Dachary <loic@dachary.org>

common: add bufferlist::rebuild_aligned_size_and_memory

The function bufferlist::rebuild_aligned checks memory and size
alignment with the same variable. It is however useful to separate
memory alignment constraints from size alignment constraints. For
instance rebuild_aligned could be called to allocate an erasure coded
buffer where each 2048 bytes chunk needs to start on a memory address
aligned on 32 bytes.

Signed-off-by: Loic Dachary <loic@dachary.org>

doc: Reformatting on rbd replay doc.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added rbd-replay-many and restructured index.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Fixed index syntax.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #3051 from ceph/wip-doc-rbd-replay

doc: Document RBD Replay

Reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #3060 from ktdreyer/doc-os-recommendations-centos-7

doc: add CentOS 7 to recommended OSes

Reviewed-by: John Wilkins <jowilkin@redhat.com>

doc: add CentOS 7 to recommended OSes

CentOS 7 was missing from the list of recommended OSes for Firefly. Add
it to the list.

Reported-By: Frank Even <lists+ceph.com@elitists.org>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #2993 from theanalyst/s3/keystone-date-fixes

rgw: check for timestamp for s3 keystone auth

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3058 from nilamdyuti/wip-doc-ceph-disk

doc: Adds man page for ceph disk in TOC.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Adds man page for ceph disk in TOC.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #3056 from ceph/wip-9997-1

client: invalidate kernel dentries one by one

Reviewed-by: John Spray <john.spray@redhat.com>

client: invalidate kernel dentries one by one

Our trick to trim the whole kernel dentry tree does not work for 3.18+ kernel.
The fix is trim kernel dentries one by one.

Fixes: #9997
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3052 from ceph/wip-flush-journal-fix

mds: fix race of trimming log segments

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #2994 from dachary/wip-10175-compile-dependencies

script replacement for deps.{rpm,deb}.txt

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Thorsten Behrens <tbehrens@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

mds: fix race of trimming log segments

trimming old log segments can start new log segments, make sure
MDLog::trim_all() not expire newly started log segments.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3041 from xiaoxichen/cleanup-noop

Cleanup noop in message

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Document RBD Replay

Signed-off-by: Adam Crume <adamcrume@gmail.com>

osd: tolerate sessionless con in fast dispatch path

We can now get a session cleared from a Connection at any time.  Change
the assert to an if in ms_fast_dispatch to cope.  It's pretty rare, but it
can happen, especially with delay injection.  In particular, a racing
thread can call mark_down() on us.

Fixes: #10209
Backport: giant
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3008 from ceph/wip-10135

mon: OSDMonitor: allow adding tiers to FS pools

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3050 from dachary/wip-10213-const

msg: do not const methods that return a mutable pointer

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

msg: do not const methods that return a mutable pointer

Add remove misleading const on post_rx_buffer argument.

http://tracker.ceph.com/issues/10213 Fixes: #10213

Signed-off-by: Loic Dachary <loic@dachary.org>

osd/osd_types: drop category from object_info_t

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: drop category from copy-get

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: drop cateogry support from CREATE

Signed-off-by: Sage Weil <sage@redhat.com>

rados: drop optional 'create [category]' arg

Signed-off-by: Sage Weil <sage@redhat.com>

rados: change 'df' output now that categories are gone

Signed-off-by: Sage Weil <sage@redhat.com>

PendingReleaseNotes: not about dropping category

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: drop useless conditionals on object_info_t decode

The incompat decode version is 8, so we will never see struct_v < 5.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: remove cat_sum from object_stat_collection_t

This type is now just a useless wrapper around object_stat_sum_t, but that
is not easy to drop.

Note that now if we load and reencode a legacy object we'll drop those
fields.  We would mark the incompat flag in the object corpus accordingly,
but nothing in there populates the cat fields.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMonitor: do not report per-category stats

Signed-off-by: Sage Weil <sage@redhat.com>

osd: stop doing per-category accounting

Signed-off-by: Sage Weil <sage@redhat.com>

librados: drop create category args

Signed-off-by: Sage Weil <sage@redhat.com>

librados: add new get_pool_stats() variant; deprecate others

A new call will return a simple map of pool -> stats.  Drop the old ones
that did pool -> category -> stats, and deprecate the category one
entirely.

Signed-off-by: Sage Weil <sage@redhat.com>

librados: mark category args and ops as unused

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2825 from ceph/wip-9921

SimpleMessenger: Pipe: do not block on takeover while holding global locks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3046 from yuyuyu101/fix-kinetic

KnieticStore: add necessary header file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2922 from ceph/wip-da-SCA-20141111

Coverity fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2937 from simon3z/master

common: do not unlock rwlock on destruction

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2945 from ceph/wip-9321

mon: PaxosService: can be readable even if proposing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2788 from ceph/wip-warn-max-pg

mon: warn when there are too many pgs per osd

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #2735 from ceph/wip-pg-stat

mon: make 'pg stat -f json' not pg dump; make JSON dump of floats floats and not strings

Reviewed-by: Samuel Just <sjust@redhat.com>

documentation: replace deps.{deb,rpm}.txt with scripts

install-deps.sh installs the dependencies based on the content of
debian/control (if apt-get is found) or ceph.spec.in (if yum is
found). The manually maintained list of dependencies for deb based
operating systems (deps.deb.txt) and rpm based operating
systems (deps.rpm.txt) are deprecated.

http://tracker.ceph.com/issues/10175 Fixes: #10175

Signed-off-by: Loic Dachary <ldachary@redhat.com>

KnieticStore: add necessary header file

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

better systemd support

Added ceph.target
Made ceph-mds, ceph-mon, ceph-osd services part of ceph.target
Made ceph-mds, ceph-mon, ceph-osd services require partitions to be available.
Added ceph init script with sysV like behaviour.

Provided by Tim Serong tserong@suse.com and Owen Synge osynge@suse.com

Signed-off-by: Owen Synge <osynge@suse.com>

Cleanup noop in message

The noop loop are dead, so clean it up

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #2855 from ceph/wip-9881

#9881 MDS flush journal asok command

Reviewed-by: Yan, Zheng <zyan@redhat.com>

mds: implement "flush journal" asok

Start a new segment and flush everything older
to the backing store.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: in segment expiry, use end instead of start

Previously we were always leaving the last log
segment un-expired from Journaler's POV, because
we were updating expire_pos to the *start* of the
expired journal segment instead of the *end*.

This was harmless, but cleaning it up makes it
easier to see what's going on during journal flushes
in testing.

Signed-off-by: John Spray <john.spray@redhat.com>