qa/workunits: cephtool: only run heap profiler test if tcmalloc enabled

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: set +e for the tcmalloc tests

Avoids failing the tests when tcmalloc is not present

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: delete unnecessary function calls

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: disable bash debug when not running tests

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: allow running individual tests

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: cleanup state after erasure-code-profile test

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: add/remove comments

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: split into properly indented functions

The test was a big sequence of commands being run and it has been growing
organically for a while, even though it has maintained a sense of
locality with regard to the portions being tested.

This patch intends to split the commands into functions, allowing for a
better semantic context and easier expansion.  On the other hand, this
will also allow us to implement mechanisms to run specific portions of
the test instead of always having to run the whole thing just to test a
couple of lines down at the bottom (or have to creatively edit the test).

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: move test line to where it's more appropriate

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunits: cephtool: split into functions

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #2021 from ceph/wip-da-fix-make-check

Fixes to get "make check" run on SUSE v2

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2001 from dachary/wip-tiermsg

mon: test that pools used in tiers cannot be removed

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #1938 from dachary/wip-8071

erasure-code: remove jerasure internals dependencies

Merge pull request #2025 from ceph/wip-disable-static

do_autogen.sh: --disable-static

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #1967 from yuyuyu101/kvstore-fix-set-alloc-hint

Make KeyValueStore not use expected_write_size

Reviewed-by: Sage Weil <sage@redhat.com>

test/ceph-disk.sh: fix for SUSE

On SUSE 'which' returns always the full path of (shell) commands and
not e.g. './ceph-conf' as on Debian. Add check also for full
path returned by which.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdmaptool/test-map-pgs.t: fix escaping to fix run

Run failed always running into the '|| cat $OUT' case due
to bad escaping of '\t'. This is caused by different shells
on different distros (e.g. bash on SUSE vs dash on Ubuntu).

Use 'grep -P ' and fix the regex to make it shell independet.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Revert "Give meaningful error when submodules are not checked out"

This reverts commit 87f9dbaf78ee040bbe0f56d2ada5299b150916b2.

This breaks the gitbuilders.  Maybe it needs the base dir to be explicit?

Signed-off-by: Sage Weil <sage@redhat.com>

Make <poolname> in "ceph osd tier --help" clearer.

The ceph osd tier --help info on the left always says <poolname>.
It is unclear which one to put <tierpool> on the right.

$ceph osd tier --help
osd tier add <poolname> <poolname> {--   add the tier <tierpool> to base pool
force-nonempty}                          <pool>
osd tier add-cache <poolname>            add a cache <tierpool> of size <size>
<poolname> <int[0-]>                     to existing pool <pool>
...

This patch modifies description on the right to tell which <poolname>:

osd tier add <poolname> <poolname> {--   add the tier <tierpool> (the second
 force-nonempty}                          one) to base pool <pool> (the first
                                           one)
...

Fix: http://tracker.ceph.com/issues/8256

Signed-off-by: Yilong Zhao <accelazh@gmail.com>

Merge pull request #1978 from adamcrume/master

OSD bug fix and developer error message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2023 from adamcrume/wip-8661

doc: Fix malformed parameters in librados.h

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #1958 from jdurgin/wip-rbd-ro

qa: add script to test krbd setting ro/rw ioctl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-mon-sanity-checks'

Reviewed-by: Sage Weil <sage@redhat.com>

mon: simplify output

Signed-off-by: Sage Weil <sage@inktank.com>

Merge commit '4225e2f6c9f69682d0e7288d7809851b61a17c70'

Conflicts:
PendingReleaseNotes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #1973 from ceph/wip-8593

mon: fix up health warnings

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #1980 from ceph/wip-osd-stats

osd: move osd_stat tracking into OSDService; fix minor bug

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2006 from ceph/wip-8603

log the command that is being run with subprocess

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #1969 from ceph/wip-vstart-conf

vstart.sh: echo CEPH_{CONF,KEYRING} exports if necessary

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2022 from ceph/wip-mon-perf

mon: shut down perfcounters last

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

do_autogen.sh: --disable-static

This will make builds go ~2x as fast when developing.

Reported-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1944 from athanatos/wip-8396

Move the waiting_for_pg list down into the Sessions, and coordinate the updates so that we don't get races
between the Pipe and handle_map threads.

Fixes: #8396
Fixes: #8505
Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #2024 from dachary/wip-8630-inject-enxio

osd: workaround race condition in tests

Reviewed-by: Sage Weil <sage@redhat.com>

osd: workaround race condition in tests

Trying to "ceph tell" a newly created OSD sometime triggers an
ENXIO. The OSD creation function used for test scripts waits for the OSD
to report that it is up (according to ceph osd tree) before
returning. That reduces (maybe close ?) the condition that triggers the
error so that the tests do not randomly fail.

http://tracker.ceph.com/issues/8630 refs: #8630

Signed-off-by: Loic Dachary <loic@dachary.org>

mon: shut down perfcounters last

In particular, remove_all_sessions updates the perfcounter stats, so we
need to stop those after.

Signed-off-by: Sage Weil <sage@redhat.com>

doc: Fix malformed parameters in librados.h

Fixes parameters for rados_write_op_operate and rados_aio_write_op_operate

Fixes: #8661
Signed-off-by: Adam Crume <adamcrume@gmail.com>

Merge pull request #1996 from dachary/wip-mailmap

mailmap updates

reviewed-by individual commits

Merge pull request #2017 from ceph/wip-8654

Fix #8654 - OSD.cc: parse LSB data via lsb_release

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

osd/OSD.cc: parse lsb release data via lsb_release

Use lsb_release tool to be portable since parsing /etc/lsb-release
is not the same between different distributions. The old code failed
e.g. for SUSE products to parse LSB information.

Fixes: #8654
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2020 from ceph/wip-fix-rados-tool

rados.cc: fix pool alignment check

Reviewed-by: Sage Weil <sage@inktank.com>

rados.cc: fix pool alignment check

Only check pool alignment if io_ctx is initialized.

Introduced in 304b08a23a3db57010078046955a786fe3589ef8
Fixes: #8652
Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge remote-tracking branch 'origin/next'

Merge pull request #2005 from ceph/wip-8624

mon: MDSMonitor: do not allow ec pools to be used for data or metadata

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: John Spray <john.spray@redhat.com>

mon: MDSMonitor: print pool name along with id during 'newfs' errors

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa/workunit: cephtool: test mds newfs and add_data_pool with ec pools

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: MDSMonitor: do not allow ec pools to be used for data or metadata

Fixes: #8624
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

common: LogClient: output to derr (i.e., dout(-1)) on CLOG_ERROR

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: Monitor: observe conf changes and report on unsage option values

Fixes: #8605
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: Monitor: sanitize options at start

Make sure some options have sane values.

Fixes: #8605
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Give meaningful error when submodules are not checked out

Signed-off-by: Adam Crume <adamcrume@gmail.com>

osd: Only normalize extent if op uses extent

Code in ReplicatedPG was reading and modifying ceph_osd_op.extent,
even though that is a member of a union and only valid for some ops.

Signed-off-by: Adam Crume <adamcrume@gmail.com>

osd: remove OSD-specific leveldb options

OSDs will now rely on 'leveldb_*' config options.  We do keep however
leveldb's log enabled for OSDs by passing 'leveldb_log=""' as a default
argument to global_init() on ceph_osd.cc -- however, users will be able
to override this at their own discretion.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

ceph-mon: override 'leveldb_*' config options for the monitor

'leveldb_*' options are currently used both by the monitor and the osd.
However, the monitor has quite different requirements from those of the
osds.

We need to specify some default values that must squash the defaults we
have for 'leveldb_*' options, while allowing users to overriding them too.
We take this not-exactly-ideal-but-still-good-enough approach of
defining the monitor-specific defaults in the 'default arguments' to
global_init(), thus allowing the user's options to take precedence over
whatever we define.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: MonitorDBStore: remove mon-specific leveldb options.

From this point onward, users should use leveldb's options and add them
to the appropriate config sections of their configuration file.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #1998 from ceph/wip-disk-ioprio

osd: allow io priority for (background) disk threadpool to be set

Backport: firefly, dumpling
Reviewed-by: Samuel Just <sam.just@inktank.com>

mon: DataHealthService: s/mon_leveldb_size_warn/mon_data_size_warn/

Make the option name independent from the store mechanism used underneath.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #2015 from ceph/wip-8610

osd: ignore CRUSH_ITEM_NONE in compat_must_dispatch_immediately

Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: ignore CRUSH_ITEM_NONE in compat_must_dispatch_immediately

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #1979 from ceph/wip-backfill-priority

osd: improve backfill prioritization

Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: allow io priority to be set for the disk_tp

The disk_tp covers scrubbing, pg deletion, and snap trimming

Signed-off-by: Sage Weil <sage@inktank.com>

common/WorkQueue: allow io priority to be set for wq

Signed-off-by: Sage Weil <sage@inktank.com>

common/Thread: allow io priority to be set for a Thread

Ideally, set this before starting the thread.  If you set it after, we
could potentially race with create() itself.

Signed-off-by: Sage Weil <sage@inktank.com>

common/io_priority: wrap ioprio_set() and gettid()

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/wip-misc-fixes'

Conflicts:
src/mon/Monitor.cc

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #2009 from kevincox/election-int

Use sized integer.

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #2011 from dachary/wip-osd-bench

osd: tests for osd bench

Reviewed-by: Huang Jun <hjwsm1989@gmail.com>

osd: tests for osd bench

Check that the safeguards against timeouts work as expected and that the
default hardcoded values are valid.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1974 from hjwsm1989/osd-bench

  Fix EINVAL err when use "ceph tell osd.* bench"

Reviewed-by: Loic Dachary <loic@dachary.org>

Use sized integer.

Signed-off-by: Kevin Cox <kevincox@kevincox.ca>

Merge pull request #1961 from ceph/wip-6703

support dmcrypt partitions when activating

Reviewed-by: Loic Dachary <loic@dachary.org>

Merge pull request #2007 from GRyall/master

doc: Fixed Typo in pools documentation - replaced '-' with '_' in eg

Reviewed-by: John Wilkins john.wilkins@inktank.com

doc: Fixed Typo in pools documentation - replaced '-' with '_' in example set-quota comands.

log the command that is being run with subprocess

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

mailmap: Dmitry Smirnov name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: koleosfuscus affiliation

Patrick McGarry, Sage Weil and Loic Dachary know who this nickname
belongs to

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Walter Huf name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Colin Mattson affliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Dominik Hannen affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Harpreet Dhillon affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Lluis Pamies-Juarez affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Wei Luo affiliation

and name normalization

Reviewed-by: Wei Luo <luowei@yahoo-inc.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Sahid Orentino Ferdjaoui name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Stephen F Taylor affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Sushma Gurram affiliation

and name normalization

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Sylvain Munaut affiliation

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Red Hat acquires InkTank

Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: Sebastien Ponce affiliation

Reviewed-by: Sebastien Ponce <sebastien.ponce@cern.ch>
Signed-off-by: Loic Dachary <loic@dachary.org>

mon: test that pools used in tiers cannot be removed

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1997 from ceph/wip-tiermsg

mon: name instead of id in "has tiers" message

Reviewed-by: Loic Dachary <loic@dachary.org>

osd/osd_types.cc: dump correct pg_log_entry_t member variable

We were dumping 'version' twice: for 'version' and 'prior_version'.  We
do have a member variable called 'prior_version', which is assumed to
be the correct one to be dumped.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

osd: use appropriate json types instead of stream when dumping info

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: name instead of id in "has tiers" message

Instead of "Pool foo has tiers 1 2" print
"Pool foo has tiers bar baz".

Signed-off-by: John Spray <jspray@redhat.com>

Merge pull request #1988 from dachary/wip-warnings

test: fix -Wsign-compare warnings

Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>

osd: ECMsgTypes: dump json bools instead of strings

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

osd: have 'tid' dumped as a json unsigned int where appropriate

Most of the places dumping it would dump it as a string. Others as an int.
Just a couple would dump it as an unsigned int.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #1993 from dachary/fix_ut

autotools: avoid check_SCRIPTS duplication

Reviewed-by: Wei Luo <luowei@yahoo-inc.com>

mon: dump 'epoch' as json unsigned instead of int

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: PGMonitor: dump 'epoch' as json int instead of string

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: OSDMonitor: dump 'up' as json array instead of string

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: OSDMonitor: dump 'acting' as json array instead of string

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>