build/ops: rpm: obsolete libcephfs1

Although under normal packaging circumstances this would probably not be done,
in Ceph there is no reasonable deployment scenario where multiple major
versions of libcephfs would need to be present at the same time.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15911 from xiexingguo/wip-bluestore-20170626

os/bluestore: narrow cache lock range; make sure min_alloc_size p2 aligned

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #16044 from liewegas/wip-i-cant-believe-its-not-btrfs

qa: stop testing btrfs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16046 from liewegas/wip-valgrind-on-centos

do all valgrind runs on centos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: recommend against btrfs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15963 from Songweibin/wip-release-notes

doc/release-notes: add Images creation timestamp note

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15924 from jcsp/wip-log-last-filters

mon: Filter `log last` output by severity and channel

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15946 from Yan-waller/wip-waller-dumpwatcher0627

osd: dump the field name of object watchers  and cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15959 from smithfarm/wip-rgw-singleton-cleanup

tests: rgw/singleton: drop duplicate filestore-xfs.yaml

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15961 from majianpeng/filestore-remove-redundant-judgement

os/filestore/FileStore.cc: remove a redundant judgement when get max latency

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #15999 from Yan-waller/wip-waller-0629message

msg/MOSDOpReply: fix missing trace decode

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16037 from gmayyyha/monmapmonitor-use-func

mon/MonmapMonitor: use __func__ instead of hard code function name

Reviewed By: Joao Eduardo Luis <jecluis@gmail.com>

do all valgrind runs on centos

We are fighting two issues with valgrind on ubuntu (xenial, yakkety,
and z):

http://tracker.ceph.com/issues/18126
http://tracker.ceph.com/issues/20360

Revert this when it is fixed.

Signed-off-by: Sage Weil <sage@redhat.com>

qa: stop testing btrfs

This is too noisy and we don't recommend it anyway.

See http://tracker.ceph.com/issues/20169

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonmapMonitor: use __func__ instead of hard code function name

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

Merge pull request #15927 from jcsp/wip-mgr-log-on-load-failed

mgr: cluster log message on plugin load error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15951 from wido/issue-20430

osd: Execute crush_location_hook when configured in ceph.conf

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15954 from Liuchang0812/wip-osd-compact

mon: fix kvstore type in mon compact command

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15957 from liewegas/wip-rc

ceph_release: we are in the 'rc' phase (12.1.z)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15968 from xiexingguo/wip-poolopts-fix

mon/OSDMonitor: two pool opts related fix

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15974 from liewegas/wip-require-osd-release-print

osd/OSDMap: print require_osd_release

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15976 from liewegas/wip-bluestore-big-cache

os/bluestore: default cache size of 3gb

Merge pull request #15489 from xxhdx1985126/master

librados: redirect balanced reads to acting primary when targeting object isn't recovered

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14678 from joscollin/wip-cleanup-storeget-return-check

mon: handle cases where store->get() may return error

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15926 from liewegas/wip-log-channel

librados: add log channel to rados_monitor_log2 callback

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15925 from jcsp/wip-fsstatus-cleanup

mgr: clean up fsstatus module

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15932 from dillaman/wip-20421

pybind: restore original API for backwards compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14760 from dzafman/wip-19657

osd: On EIO from read recover the primary replica from another copy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15912 from cloudbau/wip-20107

rgw: Fix duplicate tag removal during GC

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Tested-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #15894 from liewegas/wip-no-default-pool

mon,osd: do not create rbd pool by default

mon: make `log` command send messages to default channel

...rather than leaving channel blank.  This means that
people watching the main "cluster" channel will see
the log messages sent using `log`.

Signed-off-by: John Spray <john.spray@redhat.com>

message: fix missing decode trace

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #15471 from tchaikov/wip-radostriper-leak-on-fail

libradosstriper: fix MultiAioCompletion leaks on failure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Fix osd-scrub-repair for movement of shards

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Clean up extra json output from log

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Give time for OSDs to see map with flag change

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #15812 from dzafman/wip-cot-cleanup

tools: Cleanup dead code in ceph-objectstore-tool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15983 from cbodley/wip-20386

rgw: fix infinite loop in rest api for log list

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15635 from dzafman/wip-13837

osd: Corrupt objects stop snaptrim and mark pg snaptrim_error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15987 from liewegas/wip-19994

doc/release-notes: fix ceph-deploy command

doc/release-notes: fix ceph-deploy command

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: rest handlers for mdlog and datalog list dont loop

the do-while loops are unnecessary, because cls_log_list() will already
give us the requested number of entries, unless truncated. and these
rest operations return the truncated flag and last_marker, which the
client side is already using to run the same loop

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix RGWMetadataLog::list_entries() for null last_marker

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWMetadataLog::list_entries() no longer splices

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15850 from rzarzynski/wip-rgw-20384

rgw: fix wrong error code for expired Swift TempURL's links.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15883 from rzarzynski/wip-rgw-20395

rgw: omit X-Account-Access-Control if there is no grant to serialize.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15884 from joscollin/wip-1412617-uninitialized-pointer-field

rgw: initialize non-static class members in ESQueryCompiler

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15886 from joscollin/wip-CID1412616-uninitialized-pointer-field

rgw: initialize Non-static class member worker in RGWReshard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15888 from joscollin/wip-1412615-uninitialized-scalar-field

rgw: initialize Non-static class member val in ESQueryNodeLeafVal_Int

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15913 from smithfarm/wip-20417

tests: rgw.py: put client roles in a separate list

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test,qa/workunits: fix a zillion tests

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: do not start mapping job on empty map

The Mapper doesn't do the completion.  Just avoid it in the caller instead
of adding a Finisher or something similarly annoying for the completion.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph: create rbd pool after starting the osds

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/ceph-helpers: shorten mon connect timeout

Signed-off-by: Sage Weil <sage@redhat.com>

osdmaptool: do not create default pool without --with-default-pool

Signed-off-by: Sage Weil <sage@redhat.com>

test: update TestPGLog's pool_id for the new world (no pool 0)

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osdmap: maintain a pool-creating build_simple() for use in unit tests

Change the standard build_simple() so it no longer takes pg[p]_num -- these
params were unused. Add a new build_simple_with_pool() that can be called
when we want the rbd pool to be created by default. These are implemented
via a simple switched function (marked private).

Update the existing callers so they choose the right one for their needs.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osdmap: reference the correct value in debug output

Looks like we've been printing out the (possibly-still-default-initialized)
num_osd member variable instead of the given nosd value! Luckily
I don't see any other misuse and we fill out

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

cephtool/test: create an rbd pool since it's no longer automatic

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

TestOSDMap: provide more flexibility in the PrimaryAffinity check

Fix the margins for the primary affinity test.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>

osdmap: do not automatically create an rbd pool on cluster creation; skip pool 0

We don't use pool ID 0 any more because CephFS can't handle it.
Update the user quick-start to include it, and add some links to support that process.
Fix the tests to handle the new state of life.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

TestOSDMap: actually loop through the pools as intended

Looks like this was meant to be a loop but (unlike the other tests)
just referred to a constant pool 0. Whoops!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

pgmap: extend mon_pg_warn_min_objects to cover the PG-per-OSD warnings too

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

src/jobs: remove these definitely-broken old testing scripts

They haven't been touched in 6 years and were considered broken then.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

os/bluestore: default cache size of 3gb

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: print require_osd_release

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15967 from Adirl/continous

msg/async/rdma: register buffer as continuous

Reviewed-by: Haomai Wang <haomai@xsky.com>

fix infinite loop in rest api for log list

as max_entries_str is empty, it will be infinite loop when log is more than MAX_ENTRIES(1000) from marker.
and max_entries maybe overflow if use it to count the left entries.

Fixes: http://tracker.ceph.com/issues/20386
Signed-off-by: xierui <jerry.xr86@gmail.com>

Merge pull request #14102 from renhwztetecs/renhw-wip-moninit-cleanup

mon: clean up in ceph_mon.cc

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Jos Collin <jcollin@redhat.com>

os/bluestore: allow user to unset pool compression_mode[algorithm] properly

A string-typed pool option requires user pass in a empty string to do a valid cancellation,
but the "osd pool set " command won't allow it. E.g.:
./bin/ceph osd pool set rbd compression_mode
Invalid command:  missing required parameter val(<string>)

Since we already use the "unset" keyword to cancel the csum_type setting,
we could simply extends the above mechanism for compression_mode
and compression_algorithm too.

E.g.:
./bin/ceph osd pool set rbd compression_algorithm zlib
set pool 0 compression_algorithm to zlib

 ./bin/ceph osd pool set rbd compression_algorithm unset
unset pool 0 compression_algorithm

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/filestore/FileStore.cc: remove a redundant judgement when get max latency.

In fact, utime_t default is utime_t(). So we can remove this judgement.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

msg/async/rdma: register buffer as continuous

Change-Id: I1a715de28ace3ff8ae077ca783ad5060e7966d54
Signed-off-by: Adir Lev <adirl@mellanox.com>

os/bluestore: fix writing buffers may get pinned

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon: fix kvstore type in mon compact command

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

mon/OSDMonitor: fix error handling if we fail to set some pool opts

Returning a positive error code is generally not the preferred option
and can not be handled properly by caller here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/release-notes: add Images creation timestamp note
Signed-off-by: songweibin song.weibin@zte.com.cn

Merge pull request #15907 from tchaikov/wip-with-zfs

cmake: fix the build with -DWITH_ZFS=ON

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15908 from Adirl/twice

msg/async/rdma: use lists properly

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15521 from dillaman/wip-rbd-dashboard

pybind/mgr/dashboard: initial block integration

Reviewed-by: John Spray <john.spray@redhat.com>

tests: rgw/singleton: drop duplicate filestore-xfs.yaml

This yaml is now included in the objectstore/ directory. It makes no sense to
duplicate it here.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15853 from liewegas/wip-simpler-ceph

qa/tasks/ceph: simplify ceph deployment slightly

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph: explicitly add osds to crush map for upgrades

Before kraken, ceph-osd didn't add itself to crush... ceph-osd-prestart.sh
did it.  And ceph.py doesn't use that.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_release: we are in the 'rc' phase (12.1.z)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15955 from liewegas/wip-20431

qa/tasks/dump_stuck: fix for active+clean+remapped

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tasks/dump_stuck: fix for active+clean+remapped

In d24a8886658c2d8882275d69c6409717a62701be we made remapped a clean
state but didn't fix this test.

Fixes: http://tracker.ceph.com/issues/20431
Signed-off-by: Sage Weil <sage@redhat.com>

cmake: compile libzfs backend conditionally

* do not REQUIRE libzfs if it is enabled, this follows the way how we
  handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
  little bit.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15874 from elliswuuuuuuuu/remove_unuse_config

common/config_opt: remove unused config

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15864 from joscollin/wip-CID1412839-uninitialized-pointer-field

common: initialize array in struct BackTrace

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15881 from joscollin/wip-1412776-explicit-null-dereferenced

common: Passing null pointer option_name to operator << in md_config_t::parse_option()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15896 from Yan-waller/wip-waller-mgrcleanup0624

mgr: drop repeated log info. and unnecessary write permission

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15910 from wido/cest-api-fork

ceph-rest-api: Various REST API fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15915 from liupan1111/wip-add-perfs

src/OSD: add more useful perf counters for performance tuning.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15917 from xiexingguo/wip-upmap-fixes

mon/OSDMonitor: a couple of upmap and other fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15935 from zphj1987/master

ceph-conf: fix typo in usage: 'mon add' should be 'mon addr'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15885 from liewegas/wip-radosbench-timeout

qa/tasks/radosbench: increase timeout

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #15882 from rzarzynski/wip-20372

rgw: fixes for AWSBrowserUploadAbstractor auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15931 from cbodley/wip-rgw-slo-compression

rgw: use uncompressed size for range_to_ofs() in slo/dlo

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15928 from liewegas/wip-rgw-warning

rgw/rgw_op: fix whitespace and indentation warning

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon: handle cases where store->get() may return error

Handled cases where store->get() may return error. Fixed the review comments too in this commit.
Fixes: http://tracker.ceph.com/issues/19601
Signed-off-by: Jos Collin <jcollin@redhat.com>

osd: Execute crush_location_hook when configured in ceph.conf

CrushLocation only invoked update_from_conf() which meant that
update_from_hook() was never executed.

By called init_on_startup() both functions are executed so that
the hook is called when configured.

Signed-off-by: Wido den Hollander <wido@42on.com>