Merge pull request #2489 from guangyy/wip-9008

For pgls OP, get/put budget on per list session basis, instead of per OP basis.

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2654 from dachary/wip-9677-ioprio-class

common: ceph_ioprio_string_to_class always returns -EINVAL

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2626 from dachary/wip-rados-empty-file

tools: rados put /dev/null should write() and not create()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2652 from ceph/wip-msgr

msgr: fix remaining direct references to "new SimpleMessenger" by replacing with the factory method

Reorganize the src/msg hierarchy to push SimpleMessenger down into a subfolder to make way for organizing new in-tree implementations.

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2647 from ceph/wip-rm-legacy-cli

tools: remove old ceph.cc

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #2646 from ceph/wip-osd-op-macro

rados.h: use macros to generate osd ops definitons and strings

Reviewed-by: Sage Weil <sage@redhat.com>

Run configure without liblttng on squeeze as well.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

common: ceph_ioprio_string_to_class always returns -EINVAL

The l string is always empty because std::transform needs a
pre-allocated string. Replace with the in-place version. Add unit tests.

http://tracker.ceph.com/issues/9677 Fixes: #9677

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

osd: log error if set_ioprio fails to parse class

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

common: set_ioprio debug message including pid

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

common: do not set ioprio if pid is not set

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2653 from dachary/wip-9676-ioprio-doc

doc: missing osd prefix to ioprio config options

Reviewed-by: Mehdi Abaakouk <sileht@sileht.net>

doc: missing osd prefix to ioprio config options

http://tracker.ceph.com/issues/9676 Fixes: #9676

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

msg: move SimpleMessenger to msg/simple/

Signed-off-by: Sage Weil <sage@redhat.com>

librados: use Messenger factory method

Signed-off-by: Sage Weil <sage@redhat.com>

msg: use localized cct for derr

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.86

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2651 from ceph/wip-ostool

ceph_objectstore_tool: Accept CEPH_ARGS environment arguments

Reviewed-by: Sage Weil <sage@redhat.com>

ceph_objectstore_tool: Accept CEPH_ARGS environment arguments

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #2639 from ceph/wip-9661

Fix 9661: ceph_objectstore_tool doesn't work with memstore

Reviewed-by: Samuel Just <sam.just@inktank.com>

tools: remove old ceph.cc

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #2649 from ceph/wip-os-recommend-doc-fix

doc: update kernel recommendations, add tunables link

doc: update kernel recommendations, add tunables link

No 3.16.[0-2] - same rbd deadlock regression as in 3.15.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

msg: make messenger type configurable

Eventaully we may want to override this by passing the type in as an
arg, but let's do that later.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Message: drop unneeded #include

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: use generic Messenger factory

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2644 from objoo/master

Add one more case ( "=" ) and test get_str_vec as well.

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Add one more case ( "=" ) and test get_str_vec as well.
Signed-off-by: Yann Dupont <yann@objoo.org>

Merge pull request #2641 from dachary/wip-9655-osd-tell

qa: ceph tell must retry on ENXIO

Reviewed-by: Sage Weil <sage@redhat.com>

rados.h: use macros to generate osd ops definitons and strings

Eliminate multiple points of maintenance by using macros - no
functional changes apart from adding strings which were missing.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #2599 from johnugeorge/wip-9492-crush-indep

Crush: Ensuring at most num-rep osds are selected for indep rules

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

qa: ceph tell must retry on ENXIO

It is expected for ceph tell to fail with ENXIO if the daemon it is
trying to join is not ready for some reason. This should be handled as a
transient error instead of a fatal error.

Add two shell functions to help with retry. They may prove useful if
other cases requiring a few retries show up.

http://tracker.ceph.com/issues/9655 Fixes: #9655

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by indep rule. The fix for firstn rules is already
merged as part of bug #9492. Required test files are added.

Fixes: #9492
Signed-off-by: Johnu George johnugeo@cisco.com

Merge remote-tracking branch 'gh/giant'

Merge pull request #2640 from ceph/wip-9657

messages: provide an explicit COMPAT_VERSION in MMDSBeacon

Reviewed-by: Sage Weil <sage@redhat.com>

messages: provide an explicit COMPAT_VERSION in MMDSBeacon

We weren't setting an explicit compat version and so the Message
was setting it to the head version, breaking communication with
older daemons.

Fixes: #9657
Signed-off-by: Greg Farnum <greg@inktank.com>

MemStore: Need set/get_allow_sharded_objects() to work for ceph_objectstore_tool

Keep a bool sharded in memory and use for set/get_allow_sharded_objects()
Create a file "sharded" in the data dir when sharded is true during _save()
Check for file "sharded" during _load()

Fixes: #9661
Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_objectstore_tool: MemStore needs a CephContext

Pass g_ceph_context to ObjectStore::create() needed by MemStore

Fixes: #9661
Signed-off-by: David Zafman <dzafman@redhat.com>

debian/control: BuildDepend on lsb-release

debian/rules uses it to make lttng build conditional.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2635 from dachary/wip-9653-ceph-disk-bootstrap-osd-giant

ceph-disk: bootstrap-osd keyring ignores --statedir (giant)

Merge pull request #2633 from dachary/wip-9653-ceph-disk-bootstrap-osd

ceph-disk: bootstrap-osd keyring ignores --statedir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2637 from dachary/wip-9245-test-mon-pending-giant

mon: obsolete osdmonitor_prepare_command (giant)

tests: remove tests requiring osdmonitor_prepare_command

Although they have been useful, their maintenance and fragility caused
more trouble than their benefits.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 038de0031bc0892359201012dafb82ac1b1a4d0f)

mon: obsolete osdmonitor_prepare_command

And the mon_advanced_debug_mode option that is only used in the context
of osdmonitor_prepare_command.

http://tracker.ceph.com/issues/9245 Fixes: #9245

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 2abc8f400f7db256c2a30f1d0886a593e6c194eb)

test: minor case fix

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 6f698374562de74806203c7b41072572b59bd256)

Merge pull request #2636 from dachary/wip-9245-test-mon-pending

mon: obsolete osdmonitor_prepare_command

Reviewed-by: Sage Weil <sage@redhat.com>

tests: remove tests requiring osdmonitor_prepare_command

Although they have been useful, their maintenance and fragility caused
more trouble than their benefits.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

mon: obsolete osdmonitor_prepare_command

And the mon_advanced_debug_mode option that is only used in the context
of osdmonitor_prepare_command.

http://tracker.ceph.com/issues/9245 Fixes: #9245

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

test: minor case fix

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

ceph-disk: bootstrap-osd keyring ignores --statedir

The STATEDIR variable is used to initialize the bootstrap-osd keyring
before it gets a chance to be overriden by --statedir. Replace it with
{statedir} so that it can be substituted after all options have been
parsed.

http://tracker.ceph.com/issues/9653 Fixes: #9653

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit fa0bd06b4657e5b84e237b76033ac3d3478b6a1f)

Merge pull request #2577 from dachary/wip-9570-cpp-strerror

os: io_event.res is the size written

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>

ceph-disk: bootstrap-osd keyring ignores --statedir

The STATEDIR variable is used to initialize the bootstrap-osd keyring
before it gets a chance to be overriden by --statedir. Replace it with
{statedir} so that it can be substituted after all options have been
parsed.

http://tracker.ceph.com/issues/9653 Fixes: #9653

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2627 from theanalyst/fix/mailmap

mailmap: Yehuda Sadeh name normalization

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

mailmap: Yehuda Sadeh name normalization

Reported-by: Loic Dachary <loic@dachary.org>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

tools: rados put /dev/null should write() and not create()

In the rados.cc special case to handle put an empty objects, use
write_full() instead of create().

A special case was introduced 6843a0b81f10125842c90bc63eccc4fd873b58f2
to create() an object if the rados put file is empty. Prior to this fix
an attempt to rados put an empty file was a noop. The problem with this
fix is that it is not idempotent. rados put an empty file twice would
fail the second time and rados put a file with one byte would succeed as
expected.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2625 from ceph/wip-ysw-mailmap

mailmap: update email address

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

mailmap: update email address

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

doc/release-notes: fix attributions for 8702 fix

Oops!

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.80.6

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2623 from ceph/wip-coverity-20141002

build and coverity fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2622 from dachary/wip-osd-pool-ls

qa: fix osd pool ls invalid test

Merge pull request #2621 from ceph/wip-objecter-shutdown

Fix Objecter shutdown races with message handling

Reviewed-by: Sage Weil <sage@redhat.com>

qa: fix osd pool ls invalid test

expect_false does not extend past the pipe and fails because the command
succeeds

introduced in f05c977bbcd310a83e6df03344a72d4cc06e1cc3

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

BtrfsFileStoreBackend.cc: fix string overflow

Use strncpy() instead of strcpy and make sure the result
is '\0' terminated.

Fix for:

CID 1063700 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW)
 fixed_size_dest: You might overrun the 4088 byte fixed-size string
 vol_args.name by copying the return value of c_str without checking
 the length.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_librbd.cc: fix compiler warning, cast to size_t

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Objecter: check the 'initialized' atomic_t safely

shutdown() resets initialized to 0, but we can still receive messages
after this point, so fix message handlers to skip messages in this
case instead of asserting.

Also read initialized while holding Objecter::rwlock to avoid races
where e.g. handle_osd_map() checks initialized -> 1, continues,
shutdown() is called, sets initialized to 0, then handle_osd_map()
goes about its business and calls op_submit(), which would fail the
assert(initialized.read()) check. Similar races existed in other
message handlers which change Objecter state.

The Objecter is not destroyed until after its Messenger in
the MDS, OSD, and librados, so this should be safe.

Fixes: #9617
Backport: giant
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #2525 from ceph/wip-9487

RFC: initialize purged_snaps on backfill

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2620 from ceph/revert-2604-wip-9113

Revert "ReplicatedPG: clean out completed trimmed objects as we go"

Revert "ReplicatedPG: clean out completed trimmed objects as we go"

Merge pull request #2619 from ceph/wip-8187

librbd: Skip tier pools in list_children and snap_unprotect

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

ceph.spec: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2614 from ceph/wip-rpm-epoch

rpm: set epoch 1 to match fedora

Reviewed-by: Boris Ranto <branto@redhat.com>

librbd: Skip tier pools in list_children and snap_unprotect

Fixes: #8187
Signed-off-by: Adam Crume <adamcrume@gmail.com>

Merge branch 'wip-9013' into giant

rgw: add civetweb as a default frontend

Fixes: #9013
Originally the configurable was empty, now setting explicitly both
fastcgi and civetweb (on port 7480).

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #2552 from ceph/wip-8188

librados: Add rados_pool_get_tiers call

Reviewed-by: Sage Weil <sage@redhat.com>

librados: Add rados_pool_get_base_tier call

Fixes: #8188
Signed-off-by: Adam Crume <adamcrume@gmail.com>

Merge pull request #2618 from ceph/wip-coverity-20141001

Fixes for Coverity issues

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2452 from ceph/wip-pool-ls

mon: add 'osd pool ls [detail]' command

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

test_librbd.cc: fix compiler warning

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/SyntheticClient.cc: remove dead code

Fix for:

const: At condition do_sync, the value of do_sync must be equal to 0.
dead_error_condition: The condition do_sync cannot be true.
CID 716893 (#1 of 1): Logically dead code (DEADCODE)
dead_error_begin: Execution cannot reach this statement op.OSDOp()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

doc: A couple of fixes to the CloudStack documentation

SyntheticClient.cc: remove dead code

Fix for coverity issue:

const: At condition unsafe > 0, the value of unsafe must be equal to 0.
dead_error_condition: The condition unsafe > 0 cannot be true.

CID 716892 (#1 of 1): Logically dead code (DEADCODE)
dead_error_begin: Execution cannot reach this statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

LogEntry.h: init LogEntry::prio in constructor

CID 1240202 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member prio is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd: fix need_journal call

From 2955b3da4e2c472c18b45e81eba791f8ccd58061

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2607 from yuyuyu101/wip-9580

ObjectStore: Add "need_journal" interface to make aware of journal devic...

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2608 from yuyuyu101/pipe-iovec

Pipe: avoid redundancy new/delete for struct iovec

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.: add epoch

This is done in fedora packaging.  Do it here too so that you can move
between upstream packages (from ceph.com) and fedora and other derivatives
will builds.

Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2615 from ceph/wip-fsls-json

mon: fix JSON `fs ls` output

Reviewed-by: Sage Weil <sage@redhat.com>

mon: fix JSON `fs ls` output

A missing list terminator was causing unparseable output.

Signed-off-by: John Spray <john.spray@redhat.com>

ObjectStore: Add "need_journal" interface to make aware of journal device

Impl feature #9580

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Pipe: avoid redundancy new/delete for struct iovec

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #2564 from ceph/wip-8587

rgw: subuser creation fixes

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #2613 from ceph/wip-8822

osd: swap state spinlock for atomic_t

Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: swap state spinlock for atomic_t

We are hitting a strange issue with valgrind and pthread spinlocks. Avoid
the issue by using an atomic_t here (which is simpler anyway).

Avoids: #8822
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2612 from ceph/wip-9562-backport

osdc/Filer: drop probe/purge locks before calling objecter

Reviewed-by: Greg Farnum <greg@inktank.com>

osdc/Filer: drop probe/purge locks before calling objecter

Fixes: #9562
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8dc94a2d8ce3364c0d8d52f634e5fc967e43d819)

Locker: accept ctime updates from clients without dirty write caps

The ctime changes any time the inode does. That can happen even without
the file itself having changed, so we'd better accept the update whenever
the auth caps have dirtied, without worrying about the file caps!

Fixes: #9514
Backport: firefly

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #2605 from ceph/wip-osd-objecter

osd: do not bind ms_objecter messenger

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2597 from cchengleo/wip-test-shared-cache

test shared cache

Reviewed-by: Loic Dachary <loic-201408@dachary.org>