Merge pull request #5922 from trociny/wip-pool-set-noscrub

osd: per pool control for scrub and deep-scrub

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5846 from xiexingguo/xxg-wip-13002

msg/simple: start over after fails to bind a port in specified range

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6008 from liewegas/wip-13190

osd/ReplicatedPG: tolerate promotion completion with stopped agent

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6011 from liewegas/wip-systemd

systemd: increase nproc ulimit

Merge pull request #6019 from liewegas/wip-mon-dispatch-debug

mon/PaxosService: include msg and con ptr in dispatch debug

Merge pull request #5956 from xinxinsh/wip-13124

unify order limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5976 from yuyuyu101/wip-fix-metadata-conf

Librbd: Fix incorrect metadata filter behavior

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5750 from wonzhq/rbd-write-full

librbd: do write_full for whole object write

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5416 from zhouyuan/ceph_fuse_12204

ceph-fuse: warn and shut down when there is no MDS present

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5826 from ukernel/wip-osx-port

fix compilation on OSX/FreeBSD

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-12971' of git://github.com/jcsp/ceph

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Conflicts:
src/osdc/Objecter.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5644 from majianpeng/mds-support-iohint

Mds journal support iohint

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5966 from ukernel/wip-12506

Wip 12506

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5962 from majianpeng/client-fix

Client fix mini bugs.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon/PaxosService: include msg and con ptr in dispatch debug

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6014 from athanatos/wip-13108

Revert "osd: purge the object from the cache when proxying and not pr…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6015 from athanatos/wip-13143

osd/: find_object_context: return obc for head for snapdir if non null

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6017 from athanatos/wip-13150

ReplicatedPG::get_snapset_context: set exists to false for new ssc wi…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6013 from athanatos/wip-12828

Wip 12828

Reviewed-by: Sage Weil <sage@redhat.com>

systemd: increase nproc ulimit

We were observed to be hitting the limit on centos7
(triggering pthread_create failures) on a ~2000 OSD cluster.

Increasing this resolves it!

Reported-by: Dan van der Ster <daniel.vanderster@cern.ch>
Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: tolerate promotion completion with stopped agent

We may start a promotion, then get a pool update that disables the
agent, and then complete it.  We should not segfault in this case.

Fixes: #13190
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5918 from oritwas/wip-5378

rgw: make radosgw-admin user rm idempotent

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5996 from rzarzynski/wip-12780

rgw: don't read actual data on user manifest HEAD

Merge pull request #5974 from Sandy4999/wip-rgw-errmsg

rgw: add a new error message for user conflict when using Admin Ops API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6003 from ukernel/wip-13166

Wip 13166

Reviewed-by: John Spray <john.spray@redhat.com>

mds: adjust MDSRank::incarnation according to mdsmap

When a standby-replay MDS replace failed MDS, we need update its
incarnation.

Fixes: #13166
Signed-off-by: Yan, Zheng <zyan@redhat.com>

osd: new pool settings: scrub intervals

 ceph osd pool set $POOL scrub_min_interval N
 ceph osd pool set $POOL scrub_max_interval N
 ceph osd pool set $POOL deep_scrub_interval N

If N > 0, this value is used for the pool instead of
the corresponding global parameter from the config
(osd_scrub_min_interval, osd_scrub_max_interval or
osd_deep_scrub_interval).

Fixes: #13077
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd: new pool flags: noscrub, nodeep-scrub

This allows to disable periodic scrub on per pool basis.

Fixes: #13076
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd: make 'ceph osd pool get' work for all settable pool flags

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd: refactor setting write_fadvise_dontneed pool flag

Handle together with other flags.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6001 from dachary/wip-13182-crushtool

tests: update to match crushmap validation message

tests: update to match crushmap validation message

http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5979 from ceph/wip-crushtest

mon: fix crush testing for new pools

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5950 from tchaikov/wip-13112-infernalis

mon/PGMap: calc min_last_epoch_clean when decode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5998 from dachary/wip-13136-python3-coverage

install-deps: enable python3

install-deps: enable python3

The upstream regression has been fixed, we can re-enable python3
support.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5994 from SUSE/wip-13175

doc: do not promise backports to Dumpling

Reviewed-by: Sage Weil <sage@redhat.com>

doc: do not promise backports to Dumpling

Dumpling was retired in May 2015.

http://tracker.ceph.com/issues/13175 Fixes: #13175

Signed-off-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #5965 from jcsp/wip-get-or-create-output

mon: fix auth get-or-create output

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5984 from ceph/wip-packages

Fixes locations from ceph.com to download.ceph.com

(For the record, this is to fix http://tracker.ceph.com/issues/13162)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: remove mention of ceph-extra as a requirement

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: remove ceph-extras

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc: correct links to download.ceph.com

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mon: fix auth get-or-create output

Previously the caps were omitted from the
output: they should be present for this
to fulfil the 'get' part of get-or-create.

Signed-off-by: John Spray <john.spray@redhat.com>

ReplicatedPG::get_snapset_context: set exists to false for new ssc with can_create

Otherwise, if we do a proxy_write and don't promote it, the ssc will
stick around with exists = true and cause future reads on snaps for the
object to fail to fall through to the base tier.

Fixes: #13150
Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: don't read actual data on user manifest HEAD

Fixes: #12780
We unconditionally read all the data, which is not needed
when doing HEAD operation on user manifest objects.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ceph-osd-prestart.sh: no ceph-disk chown

chown -R ceph:ceph $1 is easy enough.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5980 from liewegas/wip-13059

ceph.spec: fix up /var/run/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: Fix up (/var)/run/ceph creation

Fixes: #13059
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5977 from tchaikov/wip-size_t-monstore-txn

mon: return size_t from MonitorDBStore::Transaction::size()

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: fix crush injection error message

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: only test crush ruleset for the newly created pool

Otherwise, we test *all* crush rules.. which might be a lot, and
which is a big waste of time and effort.

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushTester: allow testing by ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

Librbd: Fix incorrect metadata filter behavior

Signed-off-by: Haomai Wang <haomai@xsky.com>

mon: return size_t from MonitorDBStore::Transaction::size()

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add a new error message for user conflict when using Admin Ops API

Add a new error message to indicate user conflict, which was using the error code for bucket conflict.

Signed-off-by: Sangdi Xu <xu.sangdi@h3c.com>

Merge pull request #5948 from flyd1005/master

doc: delete wrong description of installing RPMs

osd/: find_object_context: return obc for head for snapdir if non null

The head obc may have an in progress operation holding a lock, we
need to block on the lock for ordering reasons.

Fixes: #13143
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #5926 from dillaman/wip-13082

include/inline_memory: out-of-bounds read on unaligned memory

Reviewed-by: Sage Weil <sage@redhat.com>

mds: Make sure wanted_state of first MDSBeacon is MDSMap::STATE_BOOT

Fixes: #12506
Signed-off-by: Yan, Zheng <zyan@redhat.com>

new release key

The previous release key,

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

may have been compromised.

The new release key is

pub   4096R/460F3994 2015-09-15
uid                  Ceph.com (release key) <security@ceph.com>

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5947 from weiqian/master

doc: replaces 'osd host' with 'host' in ceph.conf docs.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5964 from jcsp/wip-13138

libcephfs: fix calling init() then mount()

libcephfs: fix calling init() then mount()

Previously only ever called these separately, but
it should be allowed for callers to use one after
the other.

Fixes: #13138
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5963 from jcsp/wip-cmake-cephfs

CMake: fix libcephfs shared lib generation

Merge pull request #5857 from ceph/wip-da-SCA-20150812

SCA and compiler warning fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5870 from s09816/master

osd:the fuction osd::shutdown Lock failed.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5096 from tchaikov/wip-add-bl-const-iterator

common/buffer: add bufferlist::const_iterator

Reviewed-by: Sage Weil <sage@redhat.com>

CMake: fix libcephfs shared lib generation

Previously weren't generating versioned symlinks
etc, so python bindings didn't find it.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5920 from caibo2014/fix-tool-bug

tools: ceph_monstore_tool: remove the output file when get map fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5960 from dachary/wip-13136-python3-coverage

install-deps.sh: disable python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: disable python3

Disable python3 support until
https://bugs.launchpad.net/ubuntu/+source/python-coverage/+bug/1496715
is fixed.

Nothing in the build process depends on python3 right now, there is no
harm disabling it.

http://tracker.ceph.com/issues/13136 Fixes: #13136

Signed-off-by: Loic Dachary <ldachary@redhat.com>

client/MetaRequest: optimize func can_forward/auth_is_best.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

unify order limit

Fixes: #13124
Signed-off-by: xinxin shu <xinxin.shu@intel.com>

client/MetaRequest: open w/ O_CREAT|O_TRUNC is write.

From the function name is_write, it judge MetaRequest whether write.
For open, if it w/ O_CREAT|O_TRUNC, it's a write op and vice versa.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

clinet/MetaRequest: Remove the useless condition.

"head.op & CEPH_MDS_OP_WRITE" include "head.op == CEPH_MDS_OP_CREATE".
So remove the conditon statement "head.op == CEPH_MDS_OP_CREATE".

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

mon/PGMap: calc min_last_epoch_clean when decode

Fixes: #13112
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5841 from ceph/wip-12994-rbd-replay-packaging

#12994: packaging: move rbd-replay* to ceph-common

Reviewed-by: James Page <jamespage@ubuntu.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5946 from liewegas/wip-rgw-logdir

radosgw: log to /var/log/ceph instead of /var/log/radosgw

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc: delete wrong description of installing RPMs

In CentOS/RHEL 6/7, gitk and git-gui are available in default repository actually.

Signed-off-by: Li Peng <lip@dtdream.com>

doc:Replaces 'osd host' with 'host'

Signed-off-by: weiqian <weiq@dtdream.com>

radosgw: log to /var/log/ceph instead of /var/log/radosgw

This is simpler.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #5725 from liewegas/wip-scrub-stats

osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5925 from liewegas/wip-mon-leaks

fix various mon leaks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5929 from liewegas/wip-osd-leaks

misc osd leaks

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: permit MDS to send writes on full pools

Same behaviour as the global full flag, for the
per-pool full flag.  The FS was broken by 67de12bf,
which added the pool flag handling.

Fixes: #12971
Signed-off-by: John Spray <john.spray@redhat.com>

Revert "osd: purge the object from the cache when proxying and not promoting the op"

Fixes: #13108
This reverts commit da68bb371f8b06b85a73077cd069a294193ac6c0.

Merge pull request #5944 from tchaikov/wip-13090

mds: cast numbers for mds health to string when print

Reviewed-by: John Spray <john.spray@redhat.com>

osdc/Objecter: make unlocked 'full' helpers private

These never needed to be public, let's be safe.

Signed-off-by: John Spray <john.spray@redhat.com>

osdc: fix honor_osdmap_full

This was being ignored in new code that tested
the per-pool full flags.  Refactor it so that
all the tests of full flags go through _osdmap_pool_full
and thereby experience the honor_osdmap_full policy.

Fixes: #12971
Signed-off-by: John Spray <john.spray@redhat.com>

packaging: move rbd-replay* to ceph-common

The rbd-replay* utilities are useful for Ceph users with RBD clients.
Currently the rbd-replay* utilities ship in the "ceph-test" package, and
we intend this ceph-test package for Ceph developers and contributors,
not normal users.

Move the rbd-replay* utilities to "ceph-common".

http://tracker.ceph.com/issues/12994 Fixes: #12994

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

src/test/osd/Object: allow a hole at the end of the object

This way, we test the case with holes at the end of the file more
often.

Signed-off-by: Samuel Just <sjust@redhat.com>

osd_types.cc: add size to ObjectRecoveryInfo print

Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedBackend::prepare_pull: set recover_info.size for clones

Fixes: #12828
Signed-off-by: Samuel Just <sjust@redhat.com>

mon: debug refs on output replies

Signed-off-by: Sage Weil <sage@redhat.com>

msg/simple: discard local queued messages on shutdown

Otherwise these generate leak detection noise.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/Monitor: fix MonSession Leak

dispatch() does not need to keep a ref; we hold one indirectly
via the MonOpRequest.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix MonSession operator<<

Print the object, not its pointer.  Make output unambiguously
terminated.

Signed-off-by: Sage Weil <sage@redhat.com>