mon/OSDMonitor: respect NODOWN on osd failure checks

Backport: hammer, firefly
Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMonitor: avoid useless register_new_pgs work

Signed-off-by: Sage Weil <sage@redhat.com>

mon: make all paxos-related timeouts relative to mon_lease

It is a pain to adjust these settings for slow/loady clusters.
Instead, make everything a ratio of mon_lease (still default 5).

No change in defaults.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5918 from oritwas/wip-5378

rgw: make radosgw-admin user rm idempotent

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5996 from rzarzynski/wip-12780

rgw: don't read actual data on user manifest HEAD

Merge pull request #5974 from Sandy4999/wip-rgw-errmsg

rgw: add a new error message for user conflict when using Admin Ops API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6003 from ukernel/wip-13166

Wip 13166

Reviewed-by: John Spray <john.spray@redhat.com>

mds: adjust MDSRank::incarnation according to mdsmap

When a standby-replay MDS replace failed MDS, we need update its
incarnation.

Fixes: #13166
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6001 from dachary/wip-13182-crushtool

tests: update to match crushmap validation message

tests: update to match crushmap validation message

http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5979 from ceph/wip-crushtest

mon: fix crush testing for new pools

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5950 from tchaikov/wip-13112-infernalis

mon/PGMap: calc min_last_epoch_clean when decode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5998 from dachary/wip-13136-python3-coverage

install-deps: enable python3

install-deps: enable python3

The upstream regression has been fixed, we can re-enable python3
support.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5994 from SUSE/wip-13175

doc: do not promise backports to Dumpling

Reviewed-by: Sage Weil <sage@redhat.com>

doc: do not promise backports to Dumpling

Dumpling was retired in May 2015.

http://tracker.ceph.com/issues/13175 Fixes: #13175

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #5965 from jcsp/wip-get-or-create-output

mon: fix auth get-or-create output

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5984 from ceph/wip-packages

Fixes locations from ceph.com to download.ceph.com

(For the record, this is to fix http://tracker.ceph.com/issues/13162)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: remove mention of ceph-extra as a requirement

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: remove ceph-extras

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: correct links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mon: fix auth get-or-create output

Previously the caps were omitted from the
output: they should be present for this
to fulfil the 'get' part of get-or-create.

Signed-off-by: John Spray <john.spray@redhat.com>

rgw: don't read actual data on user manifest HEAD

Fixes: #12780
We unconditionally read all the data, which is not needed
when doing HEAD operation on user manifest objects.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ceph-osd-prestart.sh: no ceph-disk chown

chown -R ceph:ceph $1 is easy enough.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5980 from liewegas/wip-13059

ceph.spec: fix up /var/run/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: Fix up (/var)/run/ceph creation

Fixes: #13059
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5977 from tchaikov/wip-size_t-monstore-txn

mon: return size_t from MonitorDBStore::Transaction::size()

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: fix crush injection error message

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: only test crush ruleset for the newly created pool

Otherwise, we test *all* crush rules.. which might be a lot, and
which is a big waste of time and effort.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushTester: allow testing by ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

mon: return size_t from MonitorDBStore::Transaction::size()

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add a new error message for user conflict when using Admin Ops API

Add a new error message to indicate user conflict, which was using the error code for bucket conflict.

Signed-off-by: Sangdi Xu <xu.sangdi@h3c.com>

Merge pull request #5948 from flyd1005/master

doc: delete wrong description of installing RPMs

Merge pull request #5926 from dillaman/wip-13082

include/inline_memory: out-of-bounds read on unaligned memory

Reviewed-by: Sage Weil <sage@redhat.com>

new release key

The previous release key,

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

may have been compromised.

The new release key is

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5947 from weiqian/master

doc: replaces 'osd host' with 'host' in ceph.conf docs.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5964 from jcsp/wip-13138

libcephfs: fix calling init() then mount()

libcephfs: fix calling init() then mount()

Previously only ever called these separately, but
it should be allowed for callers to use one after
the other.

Fixes: #13138
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5963 from jcsp/wip-cmake-cephfs

CMake: fix libcephfs shared lib generation

Merge pull request #5857 from ceph/wip-da-SCA-20150812

SCA and compiler warning fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5870 from s09816/master

osd:the fuction osd::shutdown Lock failed.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5096 from tchaikov/wip-add-bl-const-iterator

common/buffer: add bufferlist::const_iterator

Reviewed-by: Sage Weil <sage@redhat.com>

CMake: fix libcephfs shared lib generation

Previously weren't generating versioned symlinks
etc, so python bindings didn't find it.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5920 from caibo2014/fix-tool-bug

tools: ceph_monstore_tool: remove the output file when get map fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5960 from dachary/wip-13136-python3-coverage

install-deps.sh: disable python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: disable python3

Disable python3 support until
https://bugs.launchpad.net/ubuntu/+source/python-coverage/+bug/1496715
is fixed.

Nothing in the build process depends on python3 right now, there is no
harm disabling it.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mon/PGMap: calc min_last_epoch_clean when decode

Fixes: #13112
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5841 from ceph/wip-12994-rbd-replay-packaging

#12994: packaging: move rbd-replay* to ceph-common

Reviewed-by: James Page <jamespage@ubuntu.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5946 from liewegas/wip-rgw-logdir

radosgw: log to /var/log/ceph instead of /var/log/radosgw

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: delete wrong description of installing RPMs

In CentOS/RHEL 6/7, gitk and git-gui are available in default repository actually.

Signed-off-by: Li Peng <lip@dtdream.com>

doc:Replaces 'osd host' with 'host'

Signed-off-by: weiqian <weiq@dtdream.com>

radosgw: log to /var/log/ceph instead of /var/log/radosgw

This is simpler.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5725 from liewegas/wip-scrub-stats

osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5925 from liewegas/wip-mon-leaks

fix various mon leaks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5929 from liewegas/wip-osd-leaks

misc osd leaks

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5944 from tchaikov/wip-13090

mds: cast numbers for mds health to string when print

Reviewed-by: John Spray <john.spray@redhat.com>

packaging: move rbd-replay* to ceph-common

The rbd-replay* utilities are useful for Ceph users with RBD clients.
Currently the rbd-replay* utilities ship in the "ceph-test" package, and
we intend this ceph-test package for Ceph developers and contributors,
not normal users.

Move the rbd-replay* utilities to "ceph-common".

http://tracker.ceph.com/issues/12994 Fixes: #12994

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

mon: debug refs on output replies

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: discard local queued messages on shutdown

Otherwise these generate leak detection noise.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Monitor: fix MonSession Leak

dispatch() does not need to keep a ref; we hold one indirectly
via the MonOpRequest.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession operator<<

Print the object, not its pointer.  Make output unambiguously
terminated.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: do not leak messages on shutdown

Signed-off-by: Sage Weil <sage@redhat.com>

mds: cast numbers for mds health to string when print

MDSHealthMetric.metadata is a map<string,string>, so the num in
`m.metadata["foo"] = num` is casted to char when being printed.
they should be coverted into std::string instead.

Fixes: #13090
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5916 from xiaoxichen/fix1

os/OSD.cc cast osd_max_write_size to int64_t

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5940 from flyd1005/master

doc: Fixes a wrong directory name and a broken hyperlink

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5912 from badone/bug_7822

examples: hello_world.cc: fix the displayed content

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5933 from dachary/wip-max-stale

common: osd_pg_epoch_persisted_max_stale < map_cache_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a broken hyperlink

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5934 from cernceph/wip-logrotate

logrotate: logs are now owned by ceph:ceph

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Fixes a wrong directory name.

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5937 from ukernel/wip-13067

Wip 13067

Reviewed-by: John Spray <john.spray@redhat.com>

Examples: hello_world.cc, content displayed after read is not null terminated.

Copy contents of bufferlist to a string before sending to stdout since a
bufferlist is not null terminated.

Fixes: #7822
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

mds: reset MDSRank heartbeat timeout even when MDS is laggy

Fixes: #13067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

common: osd_pg_epoch_persisted_max_stale < map_cache_size

1ff51a299d20dd73a95ba55fdec57498b8c71c13 reduced map_cache_size to 200,
osd_pg_epoch_persisted_max_stale must be lowered because it does not
make sense for it to be the equal or greater than map_cache_size.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

logrotate: logs are now owned by ceph:ceph

Add the su directive to avoid this error during logrotate:

error: skipping "/var/log/ceph/ceph-osd.0.log" because parent
directory has insecure permissions (It's world writable or writable
by group which is not "root") Set "su" directive in config file to
tell logrotate which user/group should be used for rotation.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

os/OSD.cc cast osd_max_write_size to int64_t

Otherwise overflow may occur and negative values generated.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os: require Sequencer arg for apply_transaction()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5923 from ceph/wip-limits

set limits in systemd unit files

ceph_objectstore_test: fix warnings

In file included from test/objectstore/store_test.cc:34:0:
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = unsigned int; T2 = int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = unsigned int; T2 = int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:411:5:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (expected == actual) {
                ^
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = unsigned int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = unsigned int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2003:7:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = long unsigned int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = long unsigned int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2010:7:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2080:5:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@redhat.com>

os/KeyValueStore: better osr debug

Signed-off-by: Sage Weil <sage@redhat.com>

os/KeyValueStore: kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

os/newstore: better sequencer debug

Signed-off-by: Sage Weil <sage@redhat.com>

os/newstore: kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: improve debug output for sequencers

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: fix leak ref on Sequencer_impl

This should start at 0 refs, not 1.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: assert Sequencer is passed in; kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

osd: queue all transactions under an osr

This was missed in e7bbafa3bfbd5e936a8be026a30b83a89f6121c3.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: eliminate ceph-rgw tmpfiles.d file

This is for storing the rgw socket files for fastcgi, which we do not
want to enable by default.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: no need to preprocess ceph-osd@service

This used to be necessary but now is not.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: set nofile limit in unit files

Make it big so hopefully nobody has to change it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop default map cache size from 500 -> 200

OSDs don't need this from each other anymore since they send
a pg_interval_map_t with pg notify.  This cache is only used
during startup when loading things up (and 200 is still > the
max advance of 150), and when clients appear with super old maps
(pretty unusual unless they are idle for long periods).

It should mean a pretty significant cut in memory...

Signed-off-by: Sage Weil <sage@redhat.com>

kill /etc/security/limits.d/ceph

This only affects login shells; not useful.

Signed-off-by: Sage Weil <sage@redhat.com>

include/inline_memory: out-of-bounds read on unaligned memory

When checking if a >=16 byte unaligned buffer is zeroed, the 32bit
check will read outside the buffer memory range.

Fixes: #13082
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tools:remove the local file when get map failed.

Signed-off-by: Bo Cai <cai.bo@h3c.com>

mon: do not return ref to MonOpRequest::get_session() caller

Most callers don't put the ref, and since we hold the op ref it's
not necessary.  This will also  make a somewhat cleaner transition
to MonSessionRef later.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession leak when waitlisting op

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: make radosgw-admin user rm idempotent

Fixes: #5378
Signed-off-by: Orit Wasserman <owasserm@redhat.com>