auth: unit tests for auid display

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

auth: add display auid to KeyServer::encode_secrets

http://tracker.ceph.com/issues/9790 Fixes: #9790

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

auth: add display auid to KeyRing::encode_formatted

http://tracker.ceph.com/issues/9790 Fixes: #9790

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2721 from wyllys66/wip-8942

librados: osd: Fixed JSON output for stray OSDs

Backport: giant, firefly
Reviewed-by: Sage Weil <sage@redhat.com>

doc/dev/sepia: notes about sepia email list and irc channels

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.80.7

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2704 from almightybeeij/master

libcephfs.h libcephfs.cc : Defined error codes for the mount function

Reviewed-by: Greg Farnum <greg@inktank.com>

doc: update lab notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #2720 from ceph/wip-5977

librbdpy: Added missing method docstrings

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librbdpy: Added missing method docstrings

Several methods were missing docstrings, preventing the methods
from appearing in the generated documentation.  Ensured all methods
now have appropriate docstrings.

Fixes: 5977
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #2718 from henrycc/wip-henrycc-inotable

mds: fix inotable initialization/reset

mds: fix inotable initialization/reset

interval_set::insert takes arguments start and len, not end.

Signed-off-by: Henry C Chang <henry.cy.chang@gmail.com>

Merge pull request #2714 from ceph/wip-9730

doc: `ceph-deploy mon create-initial` doesn't take arguments

Reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #2706 from dachary/wip-9747-ceph-spec

rpm: 95-ceph-osd-alt.rules is not needed for centos7 / rhel7

Backport: giant, firefly
Reviewed-by: Sage Weil <sage@redhat.com>

doc: remove node requirement from 'mon create-initial'

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

doc: remove whitespace

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #2591 from majianpeng/fix2

os/FileStore: using FIEMAP_FLAGS_SYNC instead of fsync() before call fiemap

Backport: giant, firefly
Reviewed-by: Sage Weil <sage@redhat.com>

FileStore:Round offset of fiemap down aligned with CEPH_PAGE_SIZE.

There is a bug on xfs about fiemap. If offset unsigned, the result of
fiemap will leak some data.
Kernel commit eedf32bfcace7d8e20cc66757d74fc68f3439ff7 fix this bug.
To avoid this bug on kernel which don't apply this commit, in ceph we
make the offset down aligned with CEPH_PAGE_SIZE.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rpm: 95-ceph-osd-alt.rules is not needed for centos7 / rhel7

The || instead of && had it always installed. That was fixed in EPEL
already.

http://tracker.ceph.com/issues/9747 Fixes: #9747

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

libcephfs.h libcephfs.cc : Defined error codes for the mount function
Used new error codes from libcephfs.h to replace the magic numbers in the mount functon found in libcephfs.cc.

Signed-off-by: BJ Lougee <almightybeeij@gmail.com>

Merge pull request #2701 from ceph/wip-9716

librados: Fix function prototypes in librados.h

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #2678 from ceph/wip-8983

rados: Parse command-line arguments strictly

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2683 from ceph/wip-7796

rgw: set length for keystone token validation request

Reviewed-by: Sage Weil <sage@redhat.com>

rados: Use strict_strtoll instead of strtoll

Signed-off-by: Adam Crume <adamcrume@gmail.com>

Merge pull request #2672 from ceph/wip-9513

Fix read performance regression in ObjectCacher

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librados: Fix function prototypes in librados.h

Fixes: 9716
Signed-off-by: Adam Crume <adamcrume@gmail.com>

rados: Add tests for CLI strict parsing

Fixes: 8983
Signed-off-by: Adam Crume <adamcrume@gmail.com>

test: Fix rmpool in test_rados_tool.sh

Signed-off-by: Adam Crume <adamcrume@gmail.com>

Merge pull request #2675 from ceph/wip-coverity-20141003

Fix issues from SCA and Coverity

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #2685 from hejiantao5/patch-1

Check the pointer before deleting

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2688 from leseb/fix-warning-message-img-format

Fix error message when stripping with format 1

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Fixed JSON output for stray OSDs

Fix error message when stripping with format 1

Since the option '--format' for specifying the rbd image format is
deprecated, we should recommend '--image-format' instead.

Signed-off-by: Sébastien Han <sebastien.han@enovance.com>

Check pointer before deleting

Here I think should check the pointer, to avoid potential risk

Signed-off-by: Jiantao He <hejiantao5@gmail.com>

ceph_erasure_code_benchmark: fix parameter handling

Make sure k and m paramter are valid to prevent crash. Fix typo.

Fix for the following CID and other possible invalid combinations
of k/m parameter:

CID 1219466 (#1 of 1): Division or modulo by zero (DIVIDE_BY_ZERO)
 divide_by_zero: In expression rand() % (k + m), modulo by expression
 k + m which may be zero has undefined behavior.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2686 from dachary/wip-client-dout

client: add missing dendl and s/dout/ldout/

Reviewed-by: John Spray <john.spray@redhat.com>

client: add missing dendl and s/dout/ldout/

Introduced in ce4436c9d4206ff4a4af7fdc16e571dcc77cd8b4

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2671 from ceph/wip-da-fix-make_check

Makefile.am: make sure everything was build before make check

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2669 from ceph/wip-9692

qa/workunits/fs/misc: fix syntax error

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2676 from ceph/wip-print-mode

client: print out mode, uid, gid if they are changed

qa/workunits/fs/misc: fix syntax error

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #2677 from ceph/wip-mds-coverity

Wip mds coverity

rgw: set length for keystone token validation request

Fixes: #7796
Backport: giany, firefly
Need to set content length to this request, as the server might not
handle a chunked request (even though we don't send anything).

Tested-by: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rados: Parse command-line arguments strictly

Many integers were parsed with functions that silently ignored junk at
the end of the string.  Extra characters now cause errors.

Fixes: 8983
Signed-off-by: Adam Crume <adamcrume@gmail.com>

mds: MDirUpdate: initialize discover count to 0 if we're not discovering

Otherwise it's left uninitialized and could do who-knows-what!

CID 1244229:  Uninitialized scalar field  (UNINIT_CTOR)
/messages/MDirUpdate.h: 55 in MDirUpdate::MDirUpdate(mds_rank_t, dirfrag_t,
int, std::set<int, std::less<int>, std::allocator<int>> &, filepath &, bool)()

Signed-off-by: Greg Farnum <greg@inktank.com>

mds: MDSAuthCaps: init "read" param in default constructor

CID 1244228:  Uninitialized scalar field  (UNINIT_CTOR)
/mds/MDSAuthCaps.h: 29 in MDSCapSpec::MDSCapSpec()()

Signed-off-by: Greg Farnum <greg@inktank.com>

client: print out mode, uid, gid if they are changed

These values can be printed by MClientRequest message output, but if
they're flushed out via cap updates we don't even get that. These
are important values that generally don't change often; so print
them out in debug logging.

Signed-off-by: Greg Farnum <greg@inktank.com>

test_rgw_admin_log.cc: refactor to use calloc()

Refactor code to use calloc() instead of malloc() to make
sure the char* TEST_BUCKET_OBJECT_SIZE memory is set to
zero before call put_bucket_obj().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_rgw_admin_log.cc: fix use-after-free

Fix for:

CID 1054876 (#1 of 1): Use after free (USE_AFTER_FREE)
 pass_freed_arg: Passing freed pointer bucket_obj as an
 argument to put_bucket_obj

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librados/c_write_operations.cc: free ressources

Release completion as soon as no longer needed.

Fix for:

CID 1219593 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable completion going out of scope
 leaks the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2511 from dachary/wip-catch-options-errors

test,tools: catch option parsing exceptions

Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

os/FileStore: Because do_fiemap will do fsync, so don't do fsync()
before calling do_fiemap.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #2629 from dachary/wip-heap-profiler

perfglue: minors heap profiler updates and documentation

Reviewed-by: Jianpeng Ma <majianpeng@gmail.com>

rados_list_parallel.cc: cleanup before return from run()

Fix for another case of:

CID 717112 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable io_ctx going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/librbd/fsx.c: fix potential unterminated string handling

In case of strncpy() make sure string is '\0' terminated. Before
usage of strcat() check if the destination has enough free space
to concat the char and to terminate the string to prevent illegal
memory access.

Fix for:

CID 1219459 (2 of 2): Buffer not null terminated (BUFFER_SIZE_WARNING)
 buffer_size_warning: Calling strncpy with a maximum size argument of
 1024 bytes on destination array goodfile of size 1024 bytes might
 leave the destination string unterminated.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeIsa.cc: reduce scope of variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SyntheticClient.cc: reduce scope of variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/flock.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedBackend.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSDMap.cc: prefer ++operator for non-primitive iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mount.ceph.c: ensure '\0' terminated string

Reserve last char in array for '\0' to ensure termination
of the string.

Fix for:

CID 1128383 (#1 of 1): Buffer not null terminated (BUFFER_SIZE_WARNING)
 buffer_size_warning: Calling strncpy with a maximum size argument
 of 1000 bytes on destination array secret of size 1000 bytes might
 leave the destination string unterminated.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #2586 from ceph/wip-mon-status

mon: make health portion of ceph -s readable (multiple lines)

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #2673 from dachary/wip-ceph-erasure-code-corpus

build: add ceph-erasure-code-corpus to gitmodule_mirrors

Reviewed-by: Sandon Van Ness <sandon@inktank.com>

build: add ceph-erasure-code-corpus to gitmodule_mirrors

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Makefile.am: make sure everything was build before make check

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge remote-tracking branch 'upstream/giant'

Merge pull request #2616 from guangyy/wip-giant-9614

PG::actingset should be used when checking the number of acting OSDs for a given PG.

Backport: firefly
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2663 from ceph/wip-9496

mon: PGMonitor: populate scrub timestamps with 'now' on pg creation

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2650 from ceph/wip-9128

Add reset_tp_timeout in long loop in add_source_info for suicide timeout

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2543 from ceph/wip-9419

Wip 9419

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #2670 from dachary/wip-9700-cephtool-mon-osd

qa: move mon_mds tests last

Reviewed-by: John Spray <john.spray@redhat.com>

qa: move mon_mds tests last

Assuming they are more likely than others to leave OSD/MON in an
unstable state that could have undefined side effects on the tests
following it. A cleaner solution would be to run them in a separate
script that is run on an independent cluster.

http://tracker.ceph.com/issues/9700 Fixes: #9700

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #2569 from ceph/wip-9437

Implement `tell` in MDS

Reviewed-by: Reviewed-by: Greg Farnum <greg@inktank.com>

qa: cephtool tests for `tell mds.X`

Signed-off-by: John Spray <john.spray@redhat.com>

qa: fixup cephtool test when MDS exists

We added MDS resetting code here a while back,
but the order of operations was such that a
"cluster up" was being run between a fail_all_mds
and the point at which we needed the map not to
be interfered with (testing setmap).

Also the new fs create/destroy cycles for testing
EC pool handling were missing calls to stop the
daemons before fs rm.

Signed-off-by: John Spray <john.spray@redhat.com>

test: unit tests for MDSAuthCaps

Signed-off-by: John Spray <john.spray@redhat.com>

vstart: create fewer pgs for fs pools

Just so that dev clusters come up more
quickly.

Signed-off-by: John Spray <john.spray@redhat.com>

vstart: mds * auth caps

...so that the default client.admin identity
can call 'tell mds' things.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: fix MDSAuthCap when cephx is disabled

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/ceph_argparse: implement 'tell mds'

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/cephfs: implement mds_command

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/cephfs: implement conf_parse_argv

So that when using `ceph` we can pass through
--debug-x args etc.

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/rados: add conf/conf_defaults attributes

Store these from initialization so that given a Rados
object, we can read back config to initialize a
LibCephFS object.

Signed-off-by: John Spray <john.spray@redhat.com>

libcephfs: add ceph_mds_command

Wraps the equivalent fn in Client.

Signed-off-by: John Spray <john.spray@redhat.com>

client: add mds_command operation

akin to Objecter::osd_command

Signed-off-by: John Spray <john.spray@redhat.com>

osdc/Objecter: don't consume non-osd cmd replies

...so that Client can get command replies from MDSs.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: introduce MDS auth caps

Signed-off-by: John Spray <john.spray@redhat.com>

mds: handle MCommand (mds tell)

Separate path to legacy MMonCommand, because MCommand
from the CLI sends JSON-style commands.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: deprecation-warn on numeric mds names

To be turned into strict check in post-hammer
release, so that we can always unambiguously
discriminate a string into one of name, rank, gid.

Signed-off-by: John Spray <john.spray@redhat.com>

common: fix cmd_getval constness

...so that one can pass in a const cmdmap.

Signed-off-by: John Spray <john.spray@redhat.com>

messages: fix constness in MCommand constructor

Signed-off-by: John Spray <john.spray@redhat.com>

mds: typedefs for rank and gid in MDSMap

Make it clearer what these numbers are where they appear.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #2668 from theanalyst/fix/mailmap-again

Fix/mailmap again

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

mailmap: João Eduardo Luís name normalization

Reported-by: Loic Dachary <loic@dachary.org>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Yongyue sun affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Andreas Peters name normalization

Reported-by: Loic Dachary <loic@dachary.org>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #2665 from yuyuyu101/update-mailmap

mailmap: add yuandong to organizationmap

Reviewed-by: Loic Dachary <loic-201408@dachary.org>

doc: update memory-profiling with examples

* Update the ceph tell from ceph daemon tell id to the new
  ceph tell deamon.id form

* Add usages examples for easier copy / paste

* Add MON to the list of daemons that can be profiled

* Document CEPH_HEAP_PROFILER_INIT=true

* Remove trailing empty lines

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

mon,mds: call ceph_heap_profiler_init() at boot time

So that the CEPH_HEAP_PROFILER_INIT environment variable can be used to
start profiling.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

perfglue: profiler stats need more than 1024 bytes

With a 1024 bytes buffer to display the stats, the end is truncated.
Use an array on the stack instead of leaking the buffer.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>