mount.ceph: avoid spurious error message

/etc/mtab in most modern distributions is a symbol link to
/proc/self/mounts.

Fixes: #10351
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3284 from ktdreyer/doc-rados-preflight-os

doc: rm reference to old Ubuntu release

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: rm reference to old Ubuntu release

Remove the reference to "Ubuntu 12.10" since this is EOL.

Clarify that we only recommend Ubuntu LTS releases.

(Since this information has a tendancy to become stale, perhaps this
whole paragraph should be removed here and we should simply point at the
main OS Recommendations page.)

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #3223 from objoo/master

mailmap updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

msg: fix factory error message

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_msgr: enable experiemntal async msgr

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3219 from yuyuyu101/wip-10383

Wip 10383: Fix unittest_msgr with AsyncMessenger bug

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3279 from ceph/wip-mon-janitorial-list

src/doc: add monitor janitorial list

Merge pull request #3278 from mslovy/wip-y04

KeyValueStore:: do_transactions: clean up code which is never used

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

KeyValueStore:: do_transactions: clean up code which is never used
clean up ops and bytes which is never used in do_transactions()

Signed-off-by: Ning Yao <zay11022@gmail.com>

src/doc: add monitor janitorial list

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3274 from ceph/wip-10341

tools: ceph-monstore-tool: refactor cli parsing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3275 from liewegas/wip-experimental

mark some features experimental and warn accordingly

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

common: log which experimental features are enabled.

This will appear in the logs regardless of whether or not the features
get used/enabled via other config options.

Signed-off-by: Sage Weil <sage@redhat.com>

ms: mark async messenger experimental

Signed-off-by: Sage Weil <sage@redhat.com>

os: rename keyvaluestore-dev -> keyvaluestore; mark experimental

Use a clean name for keyvaluestore (no -dev suffix), but mark as
experimental to ensure users know what they are signing up for.

Signed-off-by: Sage Weil <sage@redhat.com>

common: add 'enable experimental data corrupting features'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3265 from yuyuyu101/coverify-fix

Event: Avoid illegal multiplication for memset

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3260 from nilamdyuti/wip-doc-ceph-object-gateway

doc: Updates information for SSL configuration in Ceph Object Gateway installation.

Reviewed-by: John Wilkins <jowilkin@redhat.com>

tools: ceph-monstore-tool: validate start/stop vals

on commands that use 'start' and 'stop'/'end' values, we should validate
them and inform the user when 'stop' > 'start'.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: output number of paxos versions dumped

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: raw map to stdout if outfile not specified

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: use subcmd parser helper

reduces the noise in the code

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph_monstore_tool: add sub-command parser helper function

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

tools: ceph-monstore-tool: a cli that works

Fixes: #10341
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>

Event: Avoid illegal multiplication for memset

Because "file_events" is "FileEvent" type and adding an integral value to
this pointer automatically scales that value by the size.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

mailmap: Chris Holcombe affiliation

mailmap: Alfredo Deza affiliation : Fix Typo.

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Christos Stavrakakis affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Ning Yao affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Shishir Gowda affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: David Anderson affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Matt Benjamin affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

mailmap: Yunchuan Wen affiliation

Signed-off-by: Yann Dupont <yann@objoo.org>

Merge pull request #3255 from tchaikov/async-leak

AsyncMessenger: fix the leak of file_events

doc: Updates information for SSL configuration in Ceph Object Gateway installation.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

AsyncMessenger: reset newly allocated file_events

* otherwise there is chance that Event could take them as already
  initialized ones even they are filled with random bits from heap.

Merge pull request #3257 from mslovy/wip-y02

PG:: filter_snapc: Return immediately if no snapc need to trim

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3256 from majianpeng/fix4

FileJournal: Don't cache journal data if journal w/o directio mode

Reviewed-by: Sage Weil <sage@redhat.com>

PG:: filter_snapc: Return immediately if no snapc need to trim
we can return immediately if no snapc need to trim. Do not iterater snapc vector and do extra judgement and ops.

Signed-off-by: Ning Yao <zay11022@gmail.com>

FileJournal: Don't cache journal data if journal w/o directio mode

Only read journal data in osd starting.So no need to keep journal
data in kernel cache/buffer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

AsyncMessenger: fix the leak of file_events

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #3241 from ceph/wip-10415

libcephfs/test.cc: close fd before umount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3245 from trociny/fix-10132

10132: osd: tries to set ioprio when the config option is blank

Reviewed-by: Sage Weil <sage@redhat.com>

AsyncMessenger: Fix leak memory

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

10132: osd: tries to set ioprio when the config option is blank

According to documentation, ioprio params will only be used if both
osd disk thread ioprio class and osd disk thread ioprio priority are
set to a non default value.

So, add a proper check and do not generate "set_disk_tp_priority(22)
Invalid argument" warning for the default settings.

Makefile: Restore check

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Init peer_global_seq

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_msgr: Avoid deadlock between send_message and dispatch

If connection holds Connection's lock and try to acquire
FakeDispatcher's lock while gtest thread try to send_message with
FakeDispatcher's lock and try to acquire Connection's lock,
it will be deadlock.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncMessenger: Make learn_addr async to avoid destroying lock rule

Make learn_addr become a async op, otherwise holding connection's
lock then acquire messenger's lock will destroy lock rule.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection:: Avoid getting stopping connection

Now AsyncConnection::_stop may consume a little time on deleting time events,
it may occur that accepting a connection get this stopping connection
because unregister call isn't met.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Ensure reply connect_seq larger than sent

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3243 from yuyuyu101/wip-10424

librados: only call watch_flush if necessary

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librados: only call watch_flush if necessary

Fix bug #10424
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

libcephfs/test.cc: close fd before umount

Fixes: #10415
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #3200 from boydc2014/master

fix unmatched op code

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3222 from ceph/wip-watch-notify-2

librados: fix watch/notify event flushing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librados: warn about rados_watch_flush() prior to ioctx shutdown

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3114 from xiaoxichen/drop_incorrect_pool

Cleanup: Drop hobject_incorrect_pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3138 from ceph/wip-10029

SimpleMessenger: Retry binding on addresses if binding fails

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3239 from ceph/wip-stop.sh-me

stop.sh: killall -u takes username, not uid

Reviewed-by: John Spray <john.spray@redhat.com>

stop.sh: killall -u takes username, not uid

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #3183 from ceph/wip-10262

osd: fix handling of create+delete transaction

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3238 from ceph/wip-10414

client: fix uninitialized member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

client: fix uninitialized member

Fixes: #10414
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3237 from dachary/wip-10408-something-is-better-than-nothing

osd: be_compare_scrubmaps uses incorrect j iterator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3236 from ceph/wip-10409

osd: fix 10409 (object checksums for ec backends)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd: be_compare_scrubmaps uses incorrect j iterator

The code moved from be_select_auth_object to be_compare_scrubmaps
74bd8708dfbfd3c8e7ba3f41d8534609dcbc1237 but the j iterator is use
differently although it has the same type. Use map.begin() as a
fallback instead.

http://tracker.ceph.com/issues/10408 Fixes: #10408

Signed-off-by: Loic Dachary <ldachary@dachary.org>

osd: scrub: only assume shard digest == oi digest for replicated pools

For an EC object, the digest we get from scrub is for the *shard*, and that
is not the same as the *object* digest in the object_info_t.  Skip these
checks; we already have the per-shard digest that is verified in the EC
backend.

Fixes: #10409
Signed-off-by: Sage Weil <sage@redhat.com>

osd: clean up use of hex for digests

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: use correct seed for (empty) omap digest

This will be 0 on mixed OSD version PGs, -1 on current PGs.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-10409-removed-object' of git://github.com/dachary/ceph

Merge remote-tracking branch 'gh/next'

Conflicts:
PendingReleaseNotes

Merge pull request #3230 from ceph/wip-pg-stat

mon/PGMap: restructure 'pg stat' formatted output

Reviewed-by: John Spray <jspray@redhat.com>

mon/PGMap: restructure 'pg stat' formatted output

The + character, which appears in state names, is not a valid XML token.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3235 from dachary/wip-10410-port-7110

tests: use port 7111 for osd-copy-from.sh

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3234 from majianpeng/bug-fix

objectstore: add fadvise_flags operations in ObjectStore::Transaction::append func.

Reviewed-by: Sage Weil <sage@redhat.com>

remove unmatched op code comparasion

Signed-off-by: Dong Lei <leidong@yahoo-inc.com>

tests: use port 7111 for osd-copy-from.sh

Because port 7110 is already used by mkfs.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: recovery of a lost object in erasure coded pools

http://tracker.ceph.com/issues/10409 Refs: #10409

Signed-off-by: Loic Dachary <ldachary@redhat.com>

objectstore: add fadvise_flags operations in
ObjectStore::Transaction::append func.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

tests: remove spurious lines from osd-scrub-repair.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3228 from dachary/wip-10401-docker-install-deps

tests: oneliner to run make check in a container

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3231 from ceph/wip-mon-health

mon: make 'ceph -s' show pg state counts in reverse descending order

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: reduce centos docker intermediate steps

Collapse multiple RUN statements related to systemd into a single line
to reduce the number of steps that are preserved by docker.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: add docker-test.sh --ref giant|firefly etc.

Add the --ref option to chose the git ref to which the working tree must
be reset instead of HEAD.

   test/docker-test.sh --ref giant make

is equivalent to

   test/docker-test.sh bash -c 'git reset --hard giant ; make'

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: avoid bash == --shell confusion in docker-test-helper.sh

Do not force interactive mode when the bash script is specified by the
user because it should be usable to run a small script instead of
starting an interactive shell session.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: add xmlstarlet in containers

It can be removed once https://github.com/ceph/ceph/pull/3228 is merged
and xmlstarlet is found in ceph.spec.in and debian/control.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: helpers to run make check in containers

Add scripts that run make check in a container for a given operating
system version. They are a little more than oneliners and serve two
purposes:

  * help new developers who do not have to figure out which options
    should be used

  * run make check in containers in parallel with make -j2

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: run-make-check.sh arguments are passed to configure

All run-make-check.sh arguments are transparently given to the configure
command. For instance:

    run-make-check.sh --enable-root-make-check

implies

    ./configure --enable-root-make-check

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: replace --enable-docker with --enable-root-make-check

The --enable-docker logic was that each test that needs privileges to
run uses a container to do so. The problem with this approach and
make -j8 check is that such tests compete for a single container.

The --enable-root-make-check activates tests that require privileges and
assume it is ok to use sudo to acquire privileges. The decision to run
in a container is then taken by the caller who will run something like:

   docker-tests.sh ./configure --enable-root-make-check
   docker-tests.sh make -j8 check

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: docker-test.sh commands are relative to git root

Instead of being in the src directory. Forcing the working directory to
src is convenient to run unittests individually without the need to
change directory, but it is confusing to the user.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: deprecate docker-test.sh --compile option

It is redundant with the run-make-check.sh script and imposes additional
limitations.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: enable password-less sudo in containers

If compilation is run with ./configuire --enable-root-make-check, sudo
is expected to work without human interaction.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: ceph.spec.in or debian/control rebuild docker images

If any of the files in test/$os_type is newer than the creation date of
a docker image, the image is removed and rebuilt from scratch. For
instance, when a package is added to debian/control, the debian based
images are rebuilt.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: docker images must use install-deps.sh

The dockerfiles must run install-deps.sh instead of duplicating the
package list that is found in ceph.spec.in and debian/control.

A directory is created for each os_type and provided as a context for
docker build to use. The former $os_type.dockerfile is moved into
$os_type/Dockerfile.in (the .in as a reminder that it will be variable
substituted).

http://tracker.ceph.com/issues/10401 Fixes: #10401

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: run-make-check.sh install jq

jq is useful to parse json from the command line. It is however not
packaged for all supported distributions (precise has it in the backport
repository which is usually not active) and cannot be conveniently added
to debian/control because it targets all distributions and has no
conditionals.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

install-deps.sh: do not require sudo when root

If the user is root, do not use sudo. The sudo package is not installed
by default on centos by default and when building from script it may be
that root is running install-deps.sh on a freshly install distribution.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3134 from ceph/wip-9059-checksums

osd: record and validate whole-object checksums during deep scrub

Reviewed-by: Samuel Just <sjust@redhat.com>

mon: make 'ceph -s' show pg state counts in reverse descending order

Because these are in an unordered_map we print this is random order.
Instead sort descending by count.

Signed-off-by: Sage Weil <sage@redhat.com>