ceph.spec.in: Fix up (/var)/run/ceph creation

Fixes: #13059
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5948 from flyd1005/master

doc: delete wrong description of installing RPMs

Merge pull request #5926 from dillaman/wip-13082

include/inline_memory: out-of-bounds read on unaligned memory

Reviewed-by: Sage Weil <sage@redhat.com>

new release key

The previous release key,

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

may have been compromised.

The new release key is

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5947 from weiqian/master

doc: replaces 'osd host' with 'host' in ceph.conf docs.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5964 from jcsp/wip-13138

libcephfs: fix calling init() then mount()

libcephfs: fix calling init() then mount()

Previously only ever called these separately, but
it should be allowed for callers to use one after
the other.

Fixes: #13138
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5963 from jcsp/wip-cmake-cephfs

CMake: fix libcephfs shared lib generation

Merge pull request #5857 from ceph/wip-da-SCA-20150812

SCA and compiler warning fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5870 from s09816/master

osd:the fuction osd::shutdown Lock failed.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5096 from tchaikov/wip-add-bl-const-iterator

common/buffer: add bufferlist::const_iterator

Reviewed-by: Sage Weil <sage@redhat.com>

CMake: fix libcephfs shared lib generation

Previously weren't generating versioned symlinks
etc, so python bindings didn't find it.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5920 from caibo2014/fix-tool-bug

tools: ceph_monstore_tool: remove the output file when get map fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5960 from dachary/wip-13136-python3-coverage

install-deps.sh: disable python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: disable python3

Disable python3 support until
https://bugs.launchpad.net/ubuntu/+source/python-coverage/+bug/1496715
is fixed.

Nothing in the build process depends on python3 right now, there is no
harm disabling it.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5841 from ceph/wip-12994-rbd-replay-packaging

#12994: packaging: move rbd-replay* to ceph-common

Reviewed-by: James Page <jamespage@ubuntu.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5946 from liewegas/wip-rgw-logdir

radosgw: log to /var/log/ceph instead of /var/log/radosgw

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: delete wrong description of installing RPMs

In CentOS/RHEL 6/7, gitk and git-gui are available in default repository actually.

Signed-off-by: Li Peng <lip@dtdream.com>

doc:Replaces 'osd host' with 'host'

Signed-off-by: weiqian <weiq@dtdream.com>

radosgw: log to /var/log/ceph instead of /var/log/radosgw

This is simpler.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5725 from liewegas/wip-scrub-stats

osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5925 from liewegas/wip-mon-leaks

fix various mon leaks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5929 from liewegas/wip-osd-leaks

misc osd leaks

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5944 from tchaikov/wip-13090

mds: cast numbers for mds health to string when print

Reviewed-by: John Spray <john.spray@redhat.com>

packaging: move rbd-replay* to ceph-common

The rbd-replay* utilities are useful for Ceph users with RBD clients.
Currently the rbd-replay* utilities ship in the "ceph-test" package, and
we intend this ceph-test package for Ceph developers and contributors,
not normal users.

Move the rbd-replay* utilities to "ceph-common".

http://tracker.ceph.com/issues/12994 Fixes: #12994

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

mon: debug refs on output replies

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: discard local queued messages on shutdown

Otherwise these generate leak detection noise.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Monitor: fix MonSession Leak

dispatch() does not need to keep a ref; we hold one indirectly
via the MonOpRequest.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession operator<<

Print the object, not its pointer.  Make output unambiguously
terminated.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: do not leak messages on shutdown

Signed-off-by: Sage Weil <sage@redhat.com>

mds: cast numbers for mds health to string when print

MDSHealthMetric.metadata is a map<string,string>, so the num in
`m.metadata["foo"] = num` is casted to char when being printed.
they should be coverted into std::string instead.

Fixes: #13090
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5916 from xiaoxichen/fix1

os/OSD.cc cast osd_max_write_size to int64_t

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5940 from flyd1005/master

doc: Fixes a wrong directory name and a broken hyperlink

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5912 from badone/bug_7822

examples: hello_world.cc: fix the displayed content

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5933 from dachary/wip-max-stale

common: osd_pg_epoch_persisted_max_stale < map_cache_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a broken hyperlink

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5934 from cernceph/wip-logrotate

logrotate: logs are now owned by ceph:ceph

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Fixes a wrong directory name.

Signed-off-by: Li Peng <lip@dtdream.com>

Merge pull request #5937 from ukernel/wip-13067

Wip 13067

Reviewed-by: John Spray <john.spray@redhat.com>

Examples: hello_world.cc, content displayed after read is not null terminated.

Copy contents of bufferlist to a string before sending to stdout since a
bufferlist is not null terminated.

Fixes: #7822
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

mds: reset MDSRank heartbeat timeout even when MDS is laggy

Fixes: #13067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

common: osd_pg_epoch_persisted_max_stale < map_cache_size

1ff51a299d20dd73a95ba55fdec57498b8c71c13 reduced map_cache_size to 200,
osd_pg_epoch_persisted_max_stale must be lowered because it does not
make sense for it to be the equal or greater than map_cache_size.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

logrotate: logs are now owned by ceph:ceph

Add the su directive to avoid this error during logrotate:

error: skipping "/var/log/ceph/ceph-osd.0.log" because parent
directory has insecure permissions (It's world writable or writable
by group which is not "root") Set "su" directive in config file to
tell logrotate which user/group should be used for rotation.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

os/OSD.cc cast osd_max_write_size to int64_t

Otherwise overflow may occur and negative values generated.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os: require Sequencer arg for apply_transaction()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5923 from ceph/wip-limits

set limits in systemd unit files

ceph_objectstore_test: fix warnings

In file included from test/objectstore/store_test.cc:34:0:
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = unsigned int; T2 = int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = unsigned int; T2 = int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:411:5:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (expected == actual) {
                ^
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = unsigned int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = unsigned int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2003:7:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = long unsigned int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = int; T2 = long unsigned int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2010:7:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int]’:
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int; bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:2080:5:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@redhat.com>

os/KeyValueStore: better osr debug

Signed-off-by: Sage Weil <sage@redhat.com>

os/KeyValueStore: kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

os/newstore: better sequencer debug

Signed-off-by: Sage Weil <sage@redhat.com>

os/newstore: kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: improve debug output for sequencers

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: fix leak ref on Sequencer_impl

This should start at 0 refs, not 1.

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: assert Sequencer is passed in; kill default_osr

Signed-off-by: Sage Weil <sage@redhat.com>

osd: queue all transactions under an osr

This was missed in e7bbafa3bfbd5e936a8be026a30b83a89f6121c3.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: eliminate ceph-rgw tmpfiles.d file

This is for storing the rgw socket files for fastcgi, which we do not
want to enable by default.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: no need to preprocess ceph-osd@service

This used to be necessary but now is not.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: set nofile limit in unit files

Make it big so hopefully nobody has to change it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop default map cache size from 500 -> 200

OSDs don't need this from each other anymore since they send
a pg_interval_map_t with pg notify.  This cache is only used
during startup when loading things up (and 200 is still > the
max advance of 150), and when clients appear with super old maps
(pretty unusual unless they are idle for long periods).

It should mean a pretty significant cut in memory...

Signed-off-by: Sage Weil <sage@redhat.com>

kill /etc/security/limits.d/ceph

This only affects login shells; not useful.

Signed-off-by: Sage Weil <sage@redhat.com>

include/inline_memory: out-of-bounds read on unaligned memory

When checking if a >=16 byte unaligned buffer is zeroed, the 32bit
check will read outside the buffer memory range.

Fixes: #13082
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tools:remove the local file when get map failed.

Signed-off-by: Bo Cai <cai.bo@h3c.com>

mon: do not return ref to MonOpRequest::get_session() caller

Most callers don't put the ref, and since we hold the op ref it's
not necessary.  This will also  make a somewhat cleaner transition
to MonSessionRef later.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession leak when waitlisting op

Signed-off-by: Sage Weil <sage@redhat.com>

mon: debug MonSession refs

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMonitor: fix use-after-free in stats ack

The MonOpRequestRef owns one ref; we need to get() another so we
can pass it to send_reply.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: do not leak ref creating MonOpRequest

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: debug refs on sent messages

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: drop incorrect m->put()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5901 from yuyuyu101/wip-13001

AsyncMessenger: Kepp file_lock hold when accessing its event field

Merge pull request #5626 from majianpeng/objecter

Objecter: fix bus about objecter.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5840 from liewegas/wip-12991

qa/workunits/cephtool/test.sh: make mds epoch check more tolerant

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5686 from liewegas/wip-hammer-feature

require osds are hammer v0.94.4 or later before upgrading past hammer

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5905 from ceph/wip-rpm-notcmalloc

ceph.spec: respect CEPH_EXTRA_CONFIGURE_ARGS

Merge pull request #5904 from ceph/wip-submodules

fix a few submodule issues

Reviewed-by: Loic Dachary <ldachary@redhat.com>

.gitignore: ignore src/ceph.tmpe

It makes the gitbuilders unhappy.  Not sure where it comes from...

Signed-off-by: Sage Weil <sage@redhat.com>

gmock: ignore *.pyc

Signed-off-by: Sage Weil <sage@redhat.com>

rocksdb: ignore m4

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec: respect CEPH_EXTRA_CONFIGURE_ARGS

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: make mds epoch check more tolerant

This can race with an actual mdsmap epoch update for some other
reason.  We just need to make sure the epoch *increased*, not that
it is exactly old + 1.

Fixes: #12991
Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/wip-10617-again'

Merge pull request #5334 from majianpeng/read-len-zero-for-ec

make read ec-object(offset=0, len=0) can work like read no-ec object.

Reviewed-by: Kefu Chai <kchai@redhat.com>

sd/PG: tolerate missing pgmeta object

Bug 10617 left stray PG dirs around in firefly.  Hammer correctly
ignores these, assuming they are leftover cruft.  We broke this
when we dropped compat support in cd4e676e6d45c8166290ef834d73c2a0bda98fa2

See also #13060.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: allow peek_map_epoch to return an error

Allow PG::peek_map_epoch to return an error indicating the PG
should be skipped.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5776 from wonzhq/tmap-update

osd: force promote for ops which ec base pool can't handle

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd/ReplicatedPG: remove stray debug line

This snuck in

Signed-off-by: Sage Weil <sage@redhat.com>

AsyncMessenger: Kepp file_lock hold when accessing its event field

When process_event get a file event, other thread may delete this event later
but before we do event

Fix #13001
Signed-off-by: Haomai Wang <haomai@xsky.io>

Merge pull request #5896 from liewegas/wip-fix-spec

ceph.spec: include /etc/sysconfig/ceph

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ceph.spec: include /etc/sysconfig/ceph

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5895 from jcsp/wip-doc-default-pools

doc: remove references to default data/metadata pools

doc: remove references to default data/metadata pools

These haven't existed since 0.84 -- the cephfs documentation
was updated at the time, but there were also references in the
rados documentation.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5803 from ceph/wip-debian-systemd

debian systemd support; /etc/{sysconfig,default}/ceph; easy jemalloc enablement

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-common: explicitly trigger /run/ceph creation

This doesn't happen magically on debian.

Signed-off-by: Sage Weil <sage@redhat.com>

systemd: tmpfiles.d in /run, not /var/run

Signed-off-by: Sage Weil <sage@redhat.com>

debian: fix location of tmpfiles.d

/usr/lib/tmpfiles.d, not /lib/tmpfiles.d

Signed-off-by: Sage Weil <sage@redhat.com>

debian/rules: install systemd files and /etc/default/ceph

Fix the systemd units to use /etc/default instead of /etc/sysconfig.  There
is probably a better way...

Signed-off-by: Sage Weil <sage@redhat.com>

Makefile.am: include etc in tarball

Signed-off-by: Sage Weil <sage@redhat.com>