Merge pull request #3226 from ceph/wip-10319-wusui

If trusty, use older version of qemu

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3189 from ceph/wip-10329

Wip 10329

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3206 from ceph/wip-librados-init

librados: init last_objver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge remote-tracking branch 'gh/next'

Merge pull request #3247 from ceph/wip-10422

mon: provide encoded canonical full OSDMap from primary

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3291 from majianpeng/fix2

test/bufferlist: Remove chmod operation.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test/bufferlist: For root, don't do permission operation for read_file
case.

For root user, it meet those error:
    test/bufferlist.cc:1880: Failure
    Value of: bl.read_file("testfile", &error)
      Actual: 0
    Expected: -13
    test/bufferlist.cc:1884: Failure
    Value of: bl.length()
      Actual: 8
    Expected: (unsigned)4
    Which is: 4
    test/bufferlist.cc:1886: Failure
    Value of: actual
      Actual: "ABC
    ABC
    "
    Expected: "ABC\n"
    Which is: "ABC

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #3261 from tchaikov/async-msgr

AsyncMessenger: a trivial refactory and a fix

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3286 from ceph/wip-10351

mount.ceph: avoid spurious error message

Reviewed-by: Sage Weil <sage@redhat.com>

mount.ceph: avoid spurious error message

/etc/mtab in most modern distributions is a symbol link to
/proc/self/mounts.

Fixes: #10351
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3284 from ktdreyer/doc-rados-preflight-os

doc: rm reference to old Ubuntu release

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: rm reference to old Ubuntu release

Remove the reference to "Ubuntu 12.10" since this is EOL.

Clarify that we only recommend Ubuntu LTS releases.

(Since this information has a tendancy to become stale, perhaps this
whole paragraph should be removed here and we should simply point at the
main OS Recommendations page.)

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #3223 from objoo/master

mailmap updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

AsyncMessender: use IOV_MAX instead of IOV_LEN

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

AsyncMessenger: s/sended/sent/

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

AsyncMessenger: should retry in case of EINTR

* if EINTR is returned, do_send() should return 0 or `continue`.
* reword the comment a little bit

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

msg: fix factory error message

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_msgr: enable experiemntal async msgr

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3219 from yuyuyu101/wip-10383

Wip 10383: Fix unittest_msgr with AsyncMessenger bug

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3279 from ceph/wip-mon-janitorial-list

src/doc: add monitor janitorial list

Merge pull request #3278 from mslovy/wip-y04

KeyValueStore:: do_transactions: clean up code which is never used

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore:: do_transactions: clean up code which is never used
clean up ops and bytes which is never used in do_transactions()

Signed-off-by: Ning Yao <zay11022@gmail.com>

src/doc: add monitor janitorial list

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3274 from ceph/wip-10341

tools: ceph-monstore-tool: refactor cli parsing

Reviewed-by: Sage Weil <sage@redhat.com>

client: fix quota signed/unsigned warning

client/Client.cc: In member function 'bool Client::is_quota_bytes_exceeded(Inode*, uint64_t)':
client/Client.cc:10393:66: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (quota->max_bytes && (rstat->rbytes + new_bytes) > quota->max_bytes)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3275 from liewegas/wip-experimental

mark some features experimental and warn accordingly

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

common: log which experimental features are enabled.

This will appear in the logs regardless of whether or not the features
get used/enabled via other config options.

Signed-off-by: Sage Weil <sage@redhat.com>

ms: mark async messenger experimental

Signed-off-by: Sage Weil <sage@redhat.com>

os: rename keyvaluestore-dev -> keyvaluestore; mark experimental

Use a clean name for keyvaluestore (no -dev suffix), but mark as
experimental to ensure users know what they are signing up for.

Signed-off-by: Sage Weil <sage@redhat.com>

common: add 'enable experimental data corrupting features'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3265 from yuyuyu101/coverify-fix

Event: Avoid illegal multiplication for memset

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3260 from nilamdyuti/wip-doc-ceph-object-gateway

doc: Updates information for SSL configuration in Ceph Object Gateway installation.

Reviewed-by: John Wilkins <jowilkin@redhat.com>

tools: ceph-monstore-tool: validate start/stop vals

on commands that use 'start' and 'stop'/'end' values, we should validate
them and inform the user when 'stop' > 'start'.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: output number of paxos versions dumped

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: raw map to stdout if outfile not specified

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: use subcmd parser helper

reduces the noise in the code

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph_monstore_tool: add sub-command parser helper function

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: a cli that works

Fixes: #10341
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

Event: Avoid illegal multiplication for memset

Because "file_events" is "FileEvent" type and adding an integral value to
this pointer automatically scales that value by the size.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

mailmap: Chris Holcombe affiliation

mailmap: Alfredo Deza affiliation : Fix Typo.

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Christos Stavrakakis affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Ning Yao affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Shishir Gowda affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: David Anderson affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Matt Benjamin affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Yunchuan Wen affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

Merge pull request #3255 from tchaikov/async-leak

AsyncMessenger: fix the leak of file_events

doc: Updates information for SSL configuration in Ceph Object Gateway installation.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

AsyncMessenger: reset newly allocated file_events

* otherwise there is chance that Event could take them as already
  initialized ones even they are filled with random bits from heap.

Merge pull request #3257 from mslovy/wip-y02

PG:: filter_snapc: Return immediately if no snapc need to trim

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3256 from majianpeng/fix4

FileJournal: Don't cache journal data if journal w/o directio mode

Reviewed-by: Sage Weil <sage@redhat.com>

PG:: filter_snapc: Return immediately if no snapc need to trim
we can return immediately if no snapc need to trim. Do not iterater snapc vector and do extra judgement and ops.

Signed-off-by: Ning Yao <zay11022@gmail.com>

FileJournal: Don't cache journal data if journal w/o directio mode

Only read journal data in osd starting.So no need to keep journal
data in kernel cache/buffer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

AsyncMessenger: fix the leak of file_events

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mon: provide encoded canonical full OSDMap from primary

Currently we make each monitor apply the incremental and encode the full
map locally.  The original motivation was to save bandwidth, but the
savings are minimal to modest and the complexity associated with doing this
is huge.

This strategy also causes problems now that we have OSDMap crc's and old
mons/clusters may have diverging full OSDMaps due to mixed version
clusters.  See #10422

Instead, include the encoded full map in the paxos transaction.  We will
still apply the incremental and check the crc, but if it fails and we have
the correct version, reload it from disk and move on.  If we don't, we
will continue as we have before--the primary mon doesn't have support for
crc's yet.  When it does we will start verifying and/or get our
full map back into sync.

Fixes: #10422
Signed-off-by: Sage Weil <sage@redhat.com>

librados: only call watch_flush if necessary

Fix bug #10424
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
(cherry picked from commit 926a1b7468550e0c83bfa68bd45df1a6af599727)

mds: add default ctor for quota_info_t

http://tracker.ceph.com/issues/10400 Fixes: #10400

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 7f1e510165f284aad3f341ff89f64d5c3e8fe60e)

Merge pull request #3241 from ceph/wip-10415

libcephfs/test.cc: close fd before umount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3245 from trociny/fix-10132

10132: osd: tries to set ioprio when the config option is blank

Reviewed-by: Sage Weil <sage@redhat.com>

AsyncMessenger: Fix leak memory

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

10132: osd: tries to set ioprio when the config option is blank

According to documentation, ioprio params will only be used if both
osd disk thread ioprio class and osd disk thread ioprio priority are
set to a non default value.

So, add a proper check and do not generate "set_disk_tp_priority(22)
Invalid argument" warning for the default settings.

Makefile: Restore check

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Init peer_global_seq

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_msgr: Avoid deadlock between send_message and dispatch

If connection holds Connection's lock and try to acquire
FakeDispatcher's lock while gtest thread try to send_message with
FakeDispatcher's lock and try to acquire Connection's lock,
it will be deadlock.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Make learn_addr async to avoid destroying lock rule

Make learn_addr become a async op, otherwise holding connection's
lock then acquire messenger's lock will destroy lock rule.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection:: Avoid getting stopping connection

Now AsyncConnection::_stop may consume a little time on deleting time events,
it may occur that accepting a connection get this stopping connection
because unregister call isn't met.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Ensure reply connect_seq larger than sent

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3243 from yuyuyu101/wip-10424

librados: only call watch_flush if necessary

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librados: only call watch_flush if necessary

Fix bug #10424
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

libcephfs/test.cc: close fd before umount

Fixes: #10415
Signed-off-by: Yan, Zheng <zyan@redhat.com>

librados: warn about rados_watch_flush() prior to ioctx shutdown

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1fbe9b6f3a67584829fe17b6bb08cfa1b787d82c)

librados: watch_flush() on shutdown

Users can easily forget this. It makes shutdown potentially block, but if
they have racing callbacks they get what they ask for.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 30678f6daf0a71b6074c061c6b4aec6386372368)

librados: add rados_watch_flush() call

Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx.  This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4ebd4b4280cb048547842351f41b38658fb21a6e)

osdc/Objecter: do notify completion callback in fast-dispatch context

The notify completion has exactly one user, the librados caller which
does nothing but take a local (inner) lock and signal a Cond.  Do this
in the fast-dispatch context for simplicity.

Notably, this makes the notify completion (and timeout) trigger a
notify2() return (with ETIMEDOUT) even when the finisher queue that
normally delivers notify is busy.. for example with a notify that is
being very slow.  In our case, the unit test is doing a sleep(3) to
test timeouts but also prevented the ETIMEDOUT notification from
being delivered to the caller.  This patch resolves that.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b78dafd4a80ad9a557cfdf17d10fb2bff236f1a)

Merge pull request #3200 from boydc2014/master

fix unmatched op code

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3222 from ceph/wip-watch-notify-2

librados: fix watch/notify event flushing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librados: warn about rados_watch_flush() prior to ioctx shutdown

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3114 from xiaoxichen/drop_incorrect_pool

Cleanup: Drop hobject_incorrect_pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3138 from ceph/wip-10029

SimpleMessenger: Retry binding on addresses if binding fails

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3229 from ceph/wip-osd-ctor

osd/ReplicatedPG: initialize new_backfill in ctor

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3239 from ceph/wip-stop.sh-me

stop.sh: killall -u takes username, not uid

Reviewed-by: John Spray <john.spray@redhat.com>

stop.sh: killall -u takes username, not uid

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #3183 from ceph/wip-10262

osd: fix handling of create+delete transaction

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3238 from ceph/wip-10414

client: fix uninitialized member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

client: fix uninitialized member

Fixes: #10414
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3237 from dachary/wip-10408-something-is-better-than-nothing

osd: be_compare_scrubmaps uses incorrect j iterator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3236 from ceph/wip-10409

osd: fix 10409 (object checksums for ec backends)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd: be_compare_scrubmaps uses incorrect j iterator

The code moved from be_select_auth_object to be_compare_scrubmaps
74bd8708dfbfd3c8e7ba3f41d8534609dcbc1237 but the j iterator is use
differently although it has the same type. Use map.begin() as a
fallback instead.

http://tracker.ceph.com/issues/10408 Fixes: #10408

Signed-off-by: Loic Dachary <ldachary@dachary.org>

osd: scrub: only assume shard digest == oi digest for replicated pools

For an EC object, the digest we get from scrub is for the *shard*, and that
is not the same as the *object* digest in the object_info_t.  Skip these
checks; we already have the per-shard digest that is verified in the EC
backend.

Fixes: #10409
Signed-off-by: Sage Weil <sage@redhat.com>

osd: clean up use of hex for digests

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: use correct seed for (empty) omap digest

This will be 0 on mixed OSD version PGs, -1 on current PGs.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-10409-removed-object' of git://github.com/dachary/ceph

Merge remote-tracking branch 'gh/next'

Conflicts:
PendingReleaseNotes

Merge pull request #3230 from ceph/wip-pg-stat

mon/PGMap: restructure 'pg stat' formatted output

Reviewed-by: John Spray <jspray@redhat.com>

mon/PGMap: restructure 'pg stat' formatted output

The + character, which appears in state names, is not a valid XML token.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3235 from dachary/wip-10410-port-7110

tests: use port 7111 for osd-copy-from.sh

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3234 from majianpeng/bug-fix

objectstore: add fadvise_flags operations in ObjectStore::Transaction::append func.

Reviewed-by: Sage Weil <sage@redhat.com>

remove unmatched op code comparasion

Signed-off-by: Dong Lei <leidong@yahoo-inc.com>

tests: use port 7111 for osd-copy-from.sh

Because port 7110 is already used by mkfs.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>