AsyncMessenger: Remove unused option

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: add comments for structures

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: add copyright for async files

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Fix header file path

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: move Async* related file to msg/async

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Fix mark_down problem when calling send_message

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Rescheduler write event when connect successfully

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Avoid seq ack not send and skip sd<0 shutdown

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Avoid join when worker thread not started

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: mark down connection as soon as possible

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: make send_message directly send message possibly

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Reduce _try_send call

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Optimize _try_send avoid new/delete

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Asyncconnection: Fix in_seq_acked problem

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

EventEpoll: don't need to expand epoll struct

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: Use array instead of unordered_map for file_events

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: time event can used as priority queue

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: Change time precision from millisecond to microsecond

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: Fix do_request bug

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: Simply process_event impl

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Event: external_event lock may stuck in dead lock

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Avoid external thread access center

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Use round-robin to dispatch new connection

EventCenter won't own lock to protect data, because each EventCenter will
master own connections.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Fix incorrect usage for bufferlist::splice

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Rescheduler avoid dead lock

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Check send queue when message in queue before connect ready

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Use unordered_map instead of map and remove unused codes

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMsgr: Fix remote_reset segv

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Reconnect when socket is shutdown

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Use shared_ptr for EventCallback

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add wake up to EventCenter

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add support for mon library

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Replace AsyncConnection* with AsyncconnectionRef

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add EventCenter stop function

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Fix Time Event problem

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add STANDBY state process

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add TimeEvent to Event API

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add event option

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Remove DispatchQueue in AsyncMessenger

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Add AsyncMessenger support

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #2617 from ceph/wip-9628

mds: Add session to sessionmap when its state becomes opening

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge remote-tracking branch 'gh/giant'

Conflicts:
src/osdc/Objecter.cc

Merge tag 'v0.86' into giant

v0.86

0.86

Merge remote-tracking branch 'guangyy/wip-9008' into giant

Merge pull request #2631 from ceph/wip-rwtimer

DNM: osdc/Objecter: use SafeTimer; make callbacks race-tolerant

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #2489 from guangyy/wip-9008

For pgls OP, get/put budget on per list session basis, instead of per OP basis.

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2655 from ceph/wip-9635

mon: fix async write completion waits

Fixes: #9635
Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #2654 from dachary/wip-9677-ioprio-class

common: ceph_ioprio_string_to_class always returns -EINVAL

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2626 from dachary/wip-rados-empty-file

tools: rados put /dev/null should write() and not create()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2652 from ceph/wip-msgr

msgr: fix remaining direct references to "new SimpleMessenger" by replacing with the factory method

Reorganize the src/msg hierarchy to push SimpleMessenger down into a subfolder to make way for organizing new in-tree implementations.

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2560 from ceph/wip-9418

mon: add new profiles & audit cap checks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2647 from ceph/wip-rm-legacy-cli

tools: remove old ceph.cc

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #2646 from ceph/wip-osd-op-macro

rados.h: use macros to generate osd ops definitons and strings

Reviewed-by: Sage Weil <sage@redhat.com>

0.86

Run configure without liblttng on squeeze as well.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Run configure without liblttng on squeeze as well.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

common: ceph_ioprio_string_to_class always returns -EINVAL

The l string is always empty because std::transform needs a
pre-allocated string. Replace with the in-place version. Add unit tests.

http://tracker.ceph.com/issues/9677 Fixes: #9677

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

osd: log error if set_ioprio fails to parse class

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

common: set_ioprio debug message including pid

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

common: do not set ioprio if pid is not set

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2653 from dachary/wip-9676-ioprio-doc

doc: missing osd prefix to ioprio config options

Reviewed-by: Mehdi Abaakouk <sileht@sileht.net>

doc: missing osd prefix to ioprio config options

http://tracker.ceph.com/issues/9676 Fixes: #9676

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

msg: move SimpleMessenger to msg/simple/

Signed-off-by: Sage Weil <sage@redhat.com>

librados: use Messenger factory method

Signed-off-by: Sage Weil <sage@redhat.com>

msg: use localized cct for derr

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.86

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2651 from ceph/wip-ostool

ceph_objectstore_tool: Accept CEPH_ARGS environment arguments

Reviewed-by: Sage Weil <sage@redhat.com>

ceph_objectstore_tool: Accept CEPH_ARGS environment arguments

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge branch 'wip-9658' into giant

Merge pull request #2639 from ceph/wip-9661

Fix 9661: ceph_objectstore_tool doesn't work with memstore

Reviewed-by: Samuel Just <sam.just@inktank.com>

tools: remove old ceph.cc

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #2649 from ceph/wip-os-recommend-doc-fix

doc: update kernel recommendations, add tunables link

doc: update kernel recommendations, add tunables link

No 3.16.[0-2] - same rbd deadlock regression as in 3.15.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #2645 from ceph/wip-9663

client: clean-up objecter on failed client init

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec: make ceph -> python-ceph dependency specify version

It does this indirectly via ceph -> ceph-common -> python-ceph, but do so
here too for clarity/consistency.

Signed-off-by: Sage Weil <sage@redhat.com>

msg: make messenger type configurable

Eventaully we may want to override this by passing the type in as an
arg, but let's do that later.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Message: drop unneeded #include

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: use generic Messenger factory

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2642 from johnugeorge/wip-9492-crush-giant

Crush: Backporting fixes for #9492 to giant

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2644 from objoo/master

Add one more case ( "=" ) and test get_str_vec as well.

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

client: clean-up objecter on failed client init

During mount() the objecter isn't shutdown if the mon client fails to
initialize. Objecter asserts in destructor expect it to have been
shutdown but this skipped.

hadoop@plana85:~$ ./hadoop/bin/hadoop fs -ls /
14/10/05 12:35:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
osdc/Objecter.cc: In function 'virtual Objecter::~Objecter()' thread 7ff422705700 time 2014-10-05 12:35:51.090776
osdc/Objecter.cc: 3927: FAILED assert(!m_request_state_hook)
 ceph version 0.85-981-g25bcc39 (25bcc39bb809e2d13beea1529e4ab92d1b61fa5b)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0x7ff3f5c28f7f]
 2: (Objecter::~Objecter()+0x397) [0x7ff3f5bca707]
 3: (Objecter::~Objecter()+0x9) [0x7ff3f5bca8b9]
 4: (Client::~Client()+0x7d) [0x7ff3f5b6770d]
 5: (Client::~Client()+0x9) [0x7ff3f5b680a9]
 6: (ceph_mount_info::mount(std::string const&)+0x149) [0x7ff3f5b1fa49]
 7: (ceph_mount()+0x4e) [0x7ff3f5b1dcbe]
 8: (Java_com_ceph_fs_CephMount_native_1ceph_1mount()+0xb7) [0x7ff4158b1c97]
 9: [0x7ff41839dd68]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
terminate called after throwing an instance of 'ceph::FailedAssertion'
Aborted

fixes: #9663

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Add one more case ( "=" ) and test get_str_vec as well.
Signed-off-by: Yann Dupont <yann@objoo.org>

Merge pull request #2641 from dachary/wip-9655-osd-tell

qa: ceph tell must retry on ENXIO

Reviewed-by: Sage Weil <sage@redhat.com>

rados.h: use macros to generate osd ops definitons and strings

Eliminate multiple points of maintenance by using macros - no
functional changes apart from adding strings which were missing.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by indep rule. The fix for firstn rules is already
merged as part of bug #9492. Required test files are added.

Fixes: #9492
Signed-off-by: Johnu George johnugeo@cisco.com
(cherry picked from commit 234b066ba04976783d15ff2abc3e81b6cc06fb10)

Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by the rule.

Fixes: #9492
Signed-off-by: Johnu George <johnugeo@cisco.com>
(cherry picked from commit 6b4d1aa99718e3b367496326c1e64551330fabc0)

Merge pull request #2599 from johnugeorge/wip-9492-crush-indep

Crush: Ensuring at most num-rep osds are selected for indep rules

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

qa: ceph tell must retry on ENXIO

It is expected for ceph tell to fail with ENXIO if the daemon it is
trying to join is not ready for some reason. This should be handled as a
transient error instead of a fatal error.

Add two shell functions to help with retry. They may prove useful if
other cases requiring a few retries show up.

http://tracker.ceph.com/issues/9655 Fixes: #9655

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by indep rule. The fix for firstn rules is already
merged as part of bug #9492. Required test files are added.

Fixes: #9492
Signed-off-by: Johnu George johnugeo@cisco.com

Merge remote-tracking branch 'gh/giant'

Merge pull request #2640 from ceph/wip-9657

messages: provide an explicit COMPAT_VERSION in MMDSBeacon

Reviewed-by: Sage Weil <sage@redhat.com>

debian/control: fix python-ceph -> ceph file move to allow upgrades

See fe3434f41cd09433975d7d0f9dbb2fae662e4a1b.

Signed-off-by: Sage Weil <sage@redhat.com>

messages: provide an explicit COMPAT_VERSION in MMDSBeacon

We weren't setting an explicit compat version and so the Message
was setting it to the head version, breaking communication with
older daemons.

Fixes: #9657
Signed-off-by: Greg Farnum <greg@inktank.com>

MemStore: Need set/get_allow_sharded_objects() to work for ceph_objectstore_tool

Keep a bool sharded in memory and use for set/get_allow_sharded_objects()
Create a file "sharded" in the data dir when sharded is true during _save()
Check for file "sharded" during _load()

Fixes: #9661
Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_objectstore_tool: MemStore needs a CephContext

Pass g_ceph_context to ObjectStore::create() needed by MemStore

Fixes: #9661
Signed-off-by: David Zafman <dzafman@redhat.com>

debian/control: BuildDepend on lsb-release

debian/rules uses it to make lttng build conditional.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2635 from dachary/wip-9653-ceph-disk-bootstrap-osd-giant

ceph-disk: bootstrap-osd keyring ignores --statedir (giant)

Merge pull request #2633 from dachary/wip-9653-ceph-disk-bootstrap-osd

ceph-disk: bootstrap-osd keyring ignores --statedir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2637 from dachary/wip-9245-test-mon-pending-giant

mon: obsolete osdmonitor_prepare_command (giant)