use simplifed messenger constructor for clients

This is all mechanical except the calculation of the nonce, which is
now always randomized for clients.

Fixes: #13032
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

msg: add simplified messenger constructor

Several callers create messengers using exactly the same parameters:

- reading the ms type from cct that is also passed in
- a default entity_name_t::CLIENT
- the default features

Additionally, the nonce should be randomized and not depend on
e.g. pid, as it does in several callers now. Clients running in
containers can easily have pid collisions, leading to hangs, so
randomize the nonce in this simplified constructor rather than
duplicating that logic in every caller.

Daemons have meaningful entity_name_ts, and monitors currently depend
on using 0 as a nonce, so make this simple constructer
client-specific.

Related: #13032
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'wip-12971' of git://github.com/jcsp/ceph

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Conflicts:
src/osdc/Objecter.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5644 from majianpeng/mds-support-iohint

Mds journal support iohint

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5966 from ukernel/wip-12506

Wip 12506

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5962 from majianpeng/client-fix

Client fix mini bugs.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6014 from athanatos/wip-13108

Revert "osd: purge the object from the cache when proxying and not pr…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6015 from athanatos/wip-13143

osd/: find_object_context: return obc for head for snapdir if non null

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6017 from athanatos/wip-13150

ReplicatedPG::get_snapset_context: set exists to false for new ssc wi…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6013 from athanatos/wip-12828

Wip 12828

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5918 from oritwas/wip-5378

rgw: make radosgw-admin user rm idempotent

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5996 from rzarzynski/wip-12780

rgw: don't read actual data on user manifest HEAD

Merge pull request #5974 from Sandy4999/wip-rgw-errmsg

rgw: add a new error message for user conflict when using Admin Ops API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6003 from ukernel/wip-13166

Wip 13166

Reviewed-by: John Spray <john.spray@redhat.com>

mds: adjust MDSRank::incarnation according to mdsmap

When a standby-replay MDS replace failed MDS, we need update its
incarnation.

Fixes: #13166
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6001 from dachary/wip-13182-crushtool

tests: update to match crushmap validation message

tests: update to match crushmap validation message

http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5979 from ceph/wip-crushtest

mon: fix crush testing for new pools

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5950 from tchaikov/wip-13112-infernalis

mon/PGMap: calc min_last_epoch_clean when decode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5998 from dachary/wip-13136-python3-coverage

install-deps: enable python3

install-deps: enable python3

The upstream regression has been fixed, we can re-enable python3
support.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5994 from SUSE/wip-13175

doc: do not promise backports to Dumpling

Reviewed-by: Sage Weil <sage@redhat.com>

doc: do not promise backports to Dumpling

Dumpling was retired in May 2015.

http://tracker.ceph.com/issues/13175 Fixes: #13175

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #5965 from jcsp/wip-get-or-create-output

mon: fix auth get-or-create output

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5984 from ceph/wip-packages

Fixes locations from ceph.com to download.ceph.com

(For the record, this is to fix http://tracker.ceph.com/issues/13162)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: remove mention of ceph-extra as a requirement

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: remove ceph-extras

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: correct links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mon: fix auth get-or-create output

Previously the caps were omitted from the
output: they should be present for this
to fulfil the 'get' part of get-or-create.

Signed-off-by: John Spray <john.spray@redhat.com>

ReplicatedPG::get_snapset_context: set exists to false for new ssc with can_create

Otherwise, if we do a proxy_write and don't promote it, the ssc will
stick around with exists = true and cause future reads on snaps for the
object to fail to fall through to the base tier.

Fixes: #13150
Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: don't read actual data on user manifest HEAD

Fixes: #12780
We unconditionally read all the data, which is not needed
when doing HEAD operation on user manifest objects.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ceph-osd-prestart.sh: no ceph-disk chown

chown -R ceph:ceph $1 is easy enough.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5980 from liewegas/wip-13059

ceph.spec: fix up /var/run/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: Fix up (/var)/run/ceph creation

Fixes: #13059
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5977 from tchaikov/wip-size_t-monstore-txn

mon: return size_t from MonitorDBStore::Transaction::size()

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: fix crush injection error message

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: only test crush ruleset for the newly created pool

Otherwise, we test *all* crush rules.. which might be a lot, and
which is a big waste of time and effort.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushTester: allow testing by ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

mon: return size_t from MonitorDBStore::Transaction::size()

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add a new error message for user conflict when using Admin Ops API

Add a new error message to indicate user conflict, which was using the error code for bucket conflict.

Signed-off-by: Sangdi Xu <xu.sangdi@h3c.com>

Merge pull request #5948 from flyd1005/master

doc: delete wrong description of installing RPMs

osd/: find_object_context: return obc for head for snapdir if non null

The head obc may have an in progress operation holding a lock, we
need to block on the lock for ordering reasons.

Fixes: #13143
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #5926 from dillaman/wip-13082

include/inline_memory: out-of-bounds read on unaligned memory

Reviewed-by: Sage Weil <sage@redhat.com>

mds: Make sure wanted_state of first MDSBeacon is MDSMap::STATE_BOOT

Fixes: #12506
Signed-off-by: Yan, Zheng <zyan@redhat.com>

new release key

The previous release key,

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

may have been compromised.

The new release key is

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5947 from weiqian/master

doc: replaces 'osd host' with 'host' in ceph.conf docs.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5964 from jcsp/wip-13138

libcephfs: fix calling init() then mount()

libcephfs: fix calling init() then mount()

Previously only ever called these separately, but
it should be allowed for callers to use one after
the other.

Fixes: #13138
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5963 from jcsp/wip-cmake-cephfs

CMake: fix libcephfs shared lib generation

Merge pull request #5857 from ceph/wip-da-SCA-20150812

SCA and compiler warning fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5870 from s09816/master

osd:the fuction osd::shutdown Lock failed.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5096 from tchaikov/wip-add-bl-const-iterator

common/buffer: add bufferlist::const_iterator

Reviewed-by: Sage Weil <sage@redhat.com>

CMake: fix libcephfs shared lib generation

Previously weren't generating versioned symlinks
etc, so python bindings didn't find it.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5920 from caibo2014/fix-tool-bug

tools: ceph_monstore_tool: remove the output file when get map fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5960 from dachary/wip-13136-python3-coverage

install-deps.sh: disable python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: disable python3

Disable python3 support until
https://bugs.launchpad.net/ubuntu/+source/python-coverage/+bug/1496715
is fixed.

Nothing in the build process depends on python3 right now, there is no
harm disabling it.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

client/MetaRequest: optimize func can_forward/auth_is_best.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

client/MetaRequest: open w/ O_CREAT|O_TRUNC is write.

From the function name is_write, it judge MetaRequest whether write.
For open, if it w/ O_CREAT|O_TRUNC, it's a write op and vice versa.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

clinet/MetaRequest: Remove the useless condition.

"head.op & CEPH_MDS_OP_WRITE" include "head.op == CEPH_MDS_OP_CREATE".
So remove the conditon statement "head.op == CEPH_MDS_OP_CREATE".

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

mon/PGMap: calc min_last_epoch_clean when decode

Fixes: #13112
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5841 from ceph/wip-12994-rbd-replay-packaging

#12994: packaging: move rbd-replay* to ceph-common

Reviewed-by: James Page <jamespage@ubuntu.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5946 from liewegas/wip-rgw-logdir

radosgw: log to /var/log/ceph instead of /var/log/radosgw

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: delete wrong description of installing RPMs

In CentOS/RHEL 6/7, gitk and git-gui are available in default repository actually.

Signed-off-by: Li Peng <lip@dtdream.com>

doc:Replaces 'osd host' with 'host'

Signed-off-by: weiqian <weiq@dtdream.com>

radosgw: log to /var/log/ceph instead of /var/log/radosgw

This is simpler.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5725 from liewegas/wip-scrub-stats

osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5925 from liewegas/wip-mon-leaks

fix various mon leaks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5929 from liewegas/wip-osd-leaks

misc osd leaks

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: permit MDS to send writes on full pools

Same behaviour as the global full flag, for the
per-pool full flag.  The FS was broken by 67de12bf,
which added the pool flag handling.

Fixes: #12971
Signed-off-by: John Spray <john.spray@redhat.com>

Revert "osd: purge the object from the cache when proxying and not promoting the op"

Fixes: #13108
This reverts commit da68bb371f8b06b85a73077cd069a294193ac6c0.

Merge pull request #5944 from tchaikov/wip-13090

mds: cast numbers for mds health to string when print

Reviewed-by: John Spray <john.spray@redhat.com>

osdc/Objecter: make unlocked 'full' helpers private

These never needed to be public, let's be safe.

Signed-off-by: John Spray <john.spray@redhat.com>

osdc: fix honor_osdmap_full

This was being ignored in new code that tested
the per-pool full flags.  Refactor it so that
all the tests of full flags go through _osdmap_pool_full
and thereby experience the honor_osdmap_full policy.

Fixes: #12971
Signed-off-by: John Spray <john.spray@redhat.com>

packaging: move rbd-replay* to ceph-common

The rbd-replay* utilities are useful for Ceph users with RBD clients.
Currently the rbd-replay* utilities ship in the "ceph-test" package, and
we intend this ceph-test package for Ceph developers and contributors,
not normal users.

Move the rbd-replay* utilities to "ceph-common".

http://tracker.ceph.com/issues/12994 Fixes: #12994

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

src/test/osd/Object: allow a hole at the end of the object

This way, we test the case with holes at the end of the file more
often.

Signed-off-by: Samuel Just <sjust@redhat.com>

osd_types.cc: add size to ObjectRecoveryInfo print

Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedBackend::prepare_pull: set recover_info.size for clones

Fixes: #12828
Signed-off-by: Samuel Just <sjust@redhat.com>

mon: debug refs on output replies

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: discard local queued messages on shutdown

Otherwise these generate leak detection noise.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Monitor: fix MonSession Leak

dispatch() does not need to keep a ref; we hold one indirectly
via the MonOpRequest.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession operator<<

Print the object, not its pointer.  Make output unambiguously
terminated.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: do not leak messages on shutdown

Signed-off-by: Sage Weil <sage@redhat.com>

mds: cast numbers for mds health to string when print

MDSHealthMetric.metadata is a map<string,string>, so the num in
`m.metadata["foo"] = num` is casted to char when being printed.
they should be coverted into std::string instead.

Fixes: #13090
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5916 from xiaoxichen/fix1

os/OSD.cc cast osd_max_write_size to int64_t

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5940 from flyd1005/master

doc: Fixes a wrong directory name and a broken hyperlink

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5912 from badone/bug_7822

examples: hello_world.cc: fix the displayed content

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5933 from dachary/wip-max-stale

common: osd_pg_epoch_persisted_max_stale < map_cache_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a broken hyperlink

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5934 from cernceph/wip-logrotate

logrotate: logs are now owned by ceph:ceph

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Fixes a wrong directory name.

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5937 from ukernel/wip-13067

Wip 13067

Reviewed-by: John Spray <john.spray@redhat.com>

Examples: hello_world.cc, content displayed after read is not null terminated.

Copy contents of bufferlist to a string before sending to stdout since a
bufferlist is not null terminated.

Fixes: #7822
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

mds: reset MDSRank heartbeat timeout even when MDS is laggy

Fixes: #13067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

common: osd_pg_epoch_persisted_max_stale < map_cache_size

1ff51a299d20dd73a95ba55fdec57498b8c71c13 reduced map_cache_size to 200,
osd_pg_epoch_persisted_max_stale must be lowered because it does not
make sense for it to be the equal or greater than map_cache_size.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

logrotate: logs are now owned by ceph:ceph

Add the su directive to avoid this error during logrotate:

error: skipping "/var/log/ceph/ceph-osd.0.log" because parent
directory has insecure permissions (It's world writable or writable
by group which is not "root") Set "su" directive in config file to
tell logrotate which user/group should be used for rotation.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

os/OSD.cc cast osd_max_write_size to int64_t

Otherwise overflow may occur and negative values generated.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os: require Sequencer arg for apply_transaction()

Signed-off-by: Sage Weil <sage@redhat.com>