ceph.spec.in libcephfs_jni1 has no %post and %postun

/usr/lib64/libcephfs_jni.so.1.0.0 requires /sbin/ldconfig to be
run after installing and after removing.

Signed-off-by: Owen Synge <osynge@suse.com>

Merge pull request #4820 from ceph/wip-fix-ceph-perf-local

tools: fix ceph perf local build on i386

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

test/perf_local: disable tests if not supported

* pass appropriate "-m<machine>" args to compiler

Signed-off-by: Kefu Chai <kchai@redhat.com>

.gitignore: ignore /perf_perf_local

Fixes: #11827
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/next'

unittest_ceph_argparse: test float parsing

Signed-off-by: Sage Weil <sage@redhat.com>

Allow float value by allowing . in input value

Dont bail out if the input value happens to be .

Fixes: #11620
Signed-off-by: Rajesh Nambiar <rajesh.n@msystechnologies.com>
Tested-by: Sage Weil <sage@redhat.com>

Merge pull request #4810 from zaitcev/context_io_init_undup

rgw: Drop a redundant context_io.init

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: Drop a redundant context_io.init

Interestingly enough, the function is not even entirely idempotent:
it prints debugging output, duplicated.

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

Merge pull request #4812 from yuyuyu101/fix-perf-incompatible

PerfLocal: Doesnt' compile div64 for non-64 bits arch

Reviewed-by: Loic Dachary <ldachary@redhat.com>

PerfLocal: Doesnt' compile div64 for non-64 bits arch

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4797 from dachary/wip-11665-erasure-code-lrc

tests: add lrc tests to osd-scrub-repair.sh

Reviewed-by: Shylesh Kumar <shmohan@redhat.com>

Merge pull request #4799 from dachary/wip-install-deps

install-deps.sh: robust pip and wheel installation

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: do not store pip cache outside of the tree

Signed-off-by: Loic Dachary <ldachary@redhat.org>

Merge pull request #4727 from tchaikov/wip-osdmaptool-dump-tree-with-format

osdmaptool: dump tree with given format

Reviewed-by: Sage Weil <sage@redhat.com>

install-deps.sh: keep debian alternatives

Instead of removing the | when an alternative is found in the control
file such as

     cryptsetup-bin | cryptsetup

remove the surrounding spaces so that it is treated as a regular
expression that matches either one.

Replace the form

    sudo bash -c "FOO=bar ..."

with

    sudo env FOO=bar ...

to reduce the levels of shell escaping.

Reported-by: http://dyweni.com/
Signed-off-by: Loic Dachary <ldachary@redhat.com>

install-deps.sh: robust pip and wheel installation

The wheel and pip module must not only be installed in each wheelhouse
directory for tox to find. They must also be installed in the virtual
environment that populates the wheelhouse directory. Otherwise older pip
modules such as the one found by default on Ubuntu 12.04 will fail.

Python 2.7.3 on Ubuntu 12.04 also requires that distribute >= 0.7.3 is
installed although it is redundant with setuptools, otherwise it will
fail to run the wheel module.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4805 from athanatos/wip-11787

Wip 11787

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'upstream/wip-txn-noappend-sam-rebased'

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #4526 from athanatos/wip-11296

Wip 11296

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'upstream/wip-11493'

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4803 from athanatos/wip-11677

Wip 11677 -- fixes for rw rados ops with snaps

Reviewed-by: Sage Weil <sage@redhat.com>

ReplicatedPG::release_op_ctx_locks: requeue in scrub queue if blocked

Otherwise we can reorder an op around another op which got blocked by a
scrub which started after the first blocked on an obc.

Fixes: #11691
Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG::finish_ctx: take excl lock if operation is rw

Fixes: #11677
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #4804 from athanatos/wip-11771

osd_types::is_new_interval: size change triggers new interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4070 from ceph/wip-chain-xattr

os/chain_xattr: cope with xfs limit of 254 bytes for inlined xattrs.

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Ning Yao <zay11022@gmail.com>

Merge pull request #4059 from majianpeng/copyget-digest

Replicated: For COPY_GET, only src has data/omap digest, the dst can do.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3253 from majianpeng/bug-fix

cleanup code for wbthrottle.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4800 from hjwsm1989/journaler

  osdc/Journaler.h: fix ceph_file_layout dump error in journaler::dum…

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #4794 from XinzeChi/wip-kill-sleep

test: check daemon is alive or not agagin when sleep 60s

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

  osdc/Journaler.h: fix ceph_file_layout dump error in journaler::dump().
Signed-off-by: huangjun <hjwsm1989@gmail.com>

commit: test: check daemon is alive or not agagin when sleep 60s

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>

tests: add lrc tests to osd-scrub-repair.sh

Create an lrc erasure coded pool and run corruption and repair tests on
it.

http://tracker.ceph.com/issues/11665 Fixes: #11665

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: split osd-scrub-repair.sh erasure code test

The erasure code repair test is split in two function:

 * setup conditions specific to jerasure (4 osds, jerasure
   profile)

 * populate and corrupt and repair the erasure coded pool,
   with one or two failures.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: cosmetic move of functions in osd-scrub-repair.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>

erasure-code: s/unreocvery/unfound/

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4796 from SUSE/wip-release-notes-fixup

doc: release-notes.rst: fix typo and delete superfluous sentence

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: release-notes.rst: fix typo and delete superfluous sentence

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #4786 from dachary/wip-11546-debian-packages

debian/control: ceph-dbg steals ceph-objectstore-tool from ceph-test-dbg

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

debian: ceph-dbg steals ceph-objectstore-tool from ceph-test-dbg

When ceph-objectstore-tool was moved from ceph-test to
ceph by 61cf5da0b51e2d9578c7b4bca85184317e30f4ca, the ceph package in
debian/control was updated accordingly, as recommended by
https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces

The same must be done for the ceph-dbg package because
/usr/lib/debug/usr/bin/ceph-objectstore-too is no longer in
ceph-test-dbg.

Although the change was merged may 6th, 2015
8f23382064c189b657564d58c3f9d17720e891ed, teuthology jobs were not
always failing because packages were not systematically upgraded during
the installation. The missing dependencies that were responsible for
this upgrade problem were fixed by
f898ec1e4e3472b0202280f09653a769fc62c8d3 on may 18th, 2015 and all
upgrade tests relying on ceph-*-dbg packages started to fail
systematically after this date.

http://tracker.ceph.com/issues/11546 Fixes: #11546

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4787 from yuyuyu101/fix-perf-i386

PerfLocal: fix i386 compatible with perf_local

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4309 from prallabh/master

RGW: Make RADOS handles in RGW to be a configurable option

Passed rgw teuthology suite run.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4789 from yehudasa/wip-11442-notes

rgw: update release notes with regard to issue #11442 recovery

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: update release notes with regard to issue #11442 recovery

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

PerfLocal: fix i386 compatible with perf_local

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4785 from tchaikov/wip-doc-fixes

doc: typo fixes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osdmap, mon: switch the params of print_tree()

* change from
    print_tree(ostream*, Formatter*)
  to
    print_tree(Formatter*, ostream*)
  this is more consistent with other map's print
  functions. e.g.
    MDSMap::print_summary(Formatter *f, ostream *out)
* and in print_tree(Formatter* f, ostream* os),
  - `f` and `os` will be mutual exclusive.
  - will assert(0) if both of them are NULL.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdmaptool: dump 'osd tree' in specified format

* so we are able to dump more info from osdmap using this tool,
  and this allows us to reproduce the behavior of "ceph osd tree".
* add a test for 'osdmaptool --tree=<format> filepath'

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix typo in placement-groups.rst

s/osd pool set/osd pool create/ in the command line example.

Fixes: #11503
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix a wrong quote in release.rst

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4728 from branch-predictor/wip-radosbench-nomemcmp

tools: add --no-verify option to rados bench

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4547 from yuyuyu101/wip-perf

Perf: used to perf local hardware capacity

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/next'

ReplicatedPG::trim_object: write filtered snapset while we're at it

If we trimmed an object, we might as well remove the obsolete snaps
as well.

Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG: start_flush: use filtered snapset

Otherwise, we might send our deletes based on deleted snaps.  This is
problematic since we may have trimmed the clones to which those snaps
belong, causing us to send them at an earlier snap than we used before.

The specific situation was

78:[78, 70, 63, 5a, 58, 57]:[64(63), 58(58, 57)]

with 58 already clean.  To flush 64, we send:

delete@58
delete@59
copyfrom@62

Then, snap 63 is trimmed leaving us with a snapset of:

78:[78, 70, 63, 5a, 58, 57]:[58(58, 57)]

since trim_object doesn't filter the head object snapset snaps.  This
isn't really a bug since in general all snapset users must be aware
that there may be trimmed snaps in snapset::snaps.  However, here
it becomes a problem when we go to flush head:

delete@58 -- ignored due to snapc
delete@59 -- ignored due to snapc
copyfrom@78 -- not ignored

The base pool head is at snap seq 62, so it clones that value into
clone 78(78, 70) instead of forgetting it.  What should have happened
is that we should have based our flushes on filtered snapset:

78:[78, 70, 58, 57]:[58(58, 57)]

Causing us to instead send:

delete@58 -- ignored due to snapc
delete@69 -- not ignored, causes no clone to be made
copyfrom@78 -- not ignored, updates head such that a subsequent clone
will leave 70 out of the clone snaps vector.

Fixes: 11787
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #4750 from vuhuong/wip-xio

xio: several bug fixes

Merge pull request #4782 from dachary/wip-doc-ceph-helpers

tests: document ceph helpers

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4700 from ceph/wip-crush-ruleset-name

Validate default replicated ruleset config value

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4759 from yuyuyu101/wip-kvstore-misc

Wip kvstore misc

Reviewed-by: Varada Raja Kumar Kari <kariraja@yahoo.com>

tests: fix the main() documentation of ceph-helpers.sh

It was incorrectly documented as calling the setup function. It was
also vague on how the environment was reset.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: remove unused CEPH_HELPER_VERBOSE in ceph-helpers.sh

And undocumented. It may be useful for debugging but it would have to be
properly implemented instead of being half commented out.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: fix crush-ruleset-name param description

Specified crush-ruleset-name is required to exist, implicit creation is
going to happen only if crush-ruleset-name wasn't specified on the
command line.  While at it, pool-name is very much a required param.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

OSDMonitor: fix prepare_pool_crush_ruleset() retval

CrushWrapper::get_osd_pool_default_crush_replicated_ruleset() returns
-1 on error.  Returning -EPERM for "No suitable CRUSH ruleset exists"
is wrong.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

CrushWrapper: validate default replicated ruleset config opt

Validate osd_pool_default_crush_{replicated_ruleset,rule} config
options, in particular when creating pools.  Otherwise "ceph osd pool
create foo <pg_num>" may end up creating pools with non-existent
rulesets.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

OSDMap: respect default replicated ruleset config opt in build_simple()

Use id provided by osd_pool_default_crush_{replicated_ruleset,rule}
config options when creating a simple replicated ruleset for an initial
osdmap instead of always making it ruleset 0.  Not doing so may leave
default created pools (currently "rbd") in a broken state with their
crush_ruleset pointing to a non-existent ruleset.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

tests: a couple tweaks to osd-pool-create.sh

In TEST_default_deprectated_*(), make expected/unexpected vars local
and actually check that rbd, being a default created pool, is set to
use the ruleset specified by conf.

INVALIDRULESET thing in TEST_replicated_pool() is redundant - it is
checked in TEST_replicated_pool_with_ruleset() a bit earlier.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #4777 from SUSE/wip-git_ignore_systemd_autotools_template_systemd_ceph_osd

.gitignore: systemd/ceph-osd@.service

Reviewed-by: Loic Dachary <ldachary@redhat.com>

debian/copyright: update copyright for test/perf*

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4779 from theanalyst/scripts/ceph-release-notes

scripts: Add a helper to make release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

scripts: Add a helper to make release notes

First cut for a `ceph-release-notes` script added which looks at merge
commits and picks out issue numbers. Though this ideally suits for
backport releases workflow where the commit messages always follow a
specific pattern, it is partly useful for preparing release notes for
normal releases as well.

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

.gitignore: systemd/ceph-osd@.service

systemd/ceph-osd@.service is now auto-generated by autotools.

This means the file should be added to gitignored list.

Signed-off-by: Owen Synge <osynge@suse.com>

packaging: include ceph_perf_local

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

ceph_perf_local: rename from ceph_perf

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4708 from dachary/wip-erasure-code-profile

erasure-code: fix error reporting

Reviewed-by: Takeshi Miyamae <miyamae.takeshi@jp.fujitsu.com>

Merge pull request #4768 from SUSE/wip-systemd_autotools_template_systemd_ceph_osd

Template systemd/ceph-osd@.service with autotools,

Reviewed-by: Sage Weil <sage@redhat.com>

osd_types::is_new_interval: size change triggers new interval

Fixes: 11771
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #4761 from dachary/wip-doc-workflow

doc: update the development workflow

Reviewed-by: Nathan Cutler <ncutler@suse.cz>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4772 from theanalyst/doc/release/v0.94.2-updates

doc: updates to v0.94.2 draft release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4318 from ceph/wip-11056

librbd: avoid blocking AIO API methods

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: document changes to librbd's aio_read methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: update librbd AIO tests to remove result code

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: AioRequest::send no longer returns a result

The librados calls used by AioRequest::send should always return
zero unless there is a bug.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internal AIO methods no longer return result

All failures should be returned via the AioCompletion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Throttle: added pending_error method to SimpleThrottle

Allow the client of SimpleThrottle to detect an async error
so that it can exit early.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add new fail method to AioCompletion

Helper method to handle passing fatal errors generated within
librbd (not from the OSDs) back to the client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid blocking AIO API methods

Enqueue all AIO API methods within the new librbd thread pool to
reduce the possibility of any blocking operations. To maintain
backwards compatibility with the legacy return codes of the API's
AIO methods, it's still possible to block attempting to acquire
the snap_lock.

Fixes: #11056
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add task pool / work queue for requests / callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: added virtual destructor

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: add new ContextWQ work queue

The queue holds a collection of Context pointers that will
be completed by the thread pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: updates to v0.94.2 draft release notes

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

doc: update the development workflow

* use HOWTO_monitor_the_automated_tests_AKA_nightlies to explain the nightlies
* replace references to Serverity with Backport tracker
* add links to the backporter manual and the release page
* s/0.95/9.0.0/
* unify release names to be lowercase
* replace lifecycle with release cycle and end of life with retirement
* Prefer LTS or Long Term Stable over Long Term Support

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4756 from ceph/wip-mount-timeout-doc

rbd: document mount_timeout in the man page

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Template systemd/ceph-osd@.service with autotools,

The libexec path is different for different distributions.
systemd. This path is defined by a new variable on the
configure path.

This variable can be set with enviroment SYSTEMD_LIBEXEC_DIR.
The parameter --with-systemd-libexec-dir overrides the enviroment
variable.

Appropriate conditionals are set for SUSE and RHEL derivatives.

This is then used to template out systemd/ceph-osd@.service

Signed-off-by: Owen Synge <osynge@suse.com>

Merge pull request #4746 from ceph/wip-blacklist-json

osd: fix blacklist field in OSDMap::dump

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/cephtool: add blacklist json output check

...not very elegantly because this is bash, but
at least check the expected value is somewhere
present in the JSON output.

Signed-off-by: John Spray <john.spray@redhat.com>

osd: fix blacklist field in OSDMap::dump

This was using an array_section so we were getting
a list of only the times, instead of an array
mapping addr to time.

Signed-off-by: John Spray <john.spray@redhat.com>

KeyValueStore: Add collect_metadata support

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore: Avoid extra lookup for map

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4757 from xinxinsh/wip-kv-cleanup

os : remove GenericObjectMap::sync() function

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #4755 from ceph/wip-11752

mds: fix use-after-free in SessionMap::remove_session