doc/release-notes: fix ceph-deploy command

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15850 from rzarzynski/wip-rgw-20384

rgw: fix wrong error code for expired Swift TempURL's links.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15883 from rzarzynski/wip-rgw-20395

rgw: omit X-Account-Access-Control if there is no grant to serialize.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15884 from joscollin/wip-1412617-uninitialized-pointer-field

rgw: initialize non-static class members in ESQueryCompiler

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15886 from joscollin/wip-CID1412616-uninitialized-pointer-field

rgw: initialize Non-static class member worker in RGWReshard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15888 from joscollin/wip-1412615-uninitialized-scalar-field

rgw: initialize Non-static class member val in ESQueryNodeLeafVal_Int

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15913 from smithfarm/wip-20417

tests: rgw.py: put client roles in a separate list

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15967 from Adirl/continous

msg/async/rdma: register buffer as continuous

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14102 from renhwztetecs/renhw-wip-moninit-cleanup

mon: clean up in ceph_mon.cc

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Jos Collin <jcollin@redhat.com>

msg/async/rdma: register buffer as continuous

Change-Id: I1a715de28ace3ff8ae077ca783ad5060e7966d54
Signed-off-by: Adir Lev <adirl@mellanox.com>

Merge pull request #15907 from tchaikov/wip-with-zfs

cmake: fix the build with -DWITH_ZFS=ON

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15908 from Adirl/twice

msg/async/rdma: use lists properly

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15521 from dillaman/wip-rbd-dashboard

pybind/mgr/dashboard: initial block integration

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15853 from liewegas/wip-simpler-ceph

qa/tasks/ceph: simplify ceph deployment slightly

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph: explicitly add osds to crush map for upgrades

Before kraken, ceph-osd didn't add itself to crush... ceph-osd-prestart.sh
did it.  And ceph.py doesn't use that.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15955 from liewegas/wip-20431

qa/tasks/dump_stuck: fix for active+clean+remapped

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tasks/dump_stuck: fix for active+clean+remapped

In d24a8886658c2d8882275d69c6409717a62701be we made remapped a clean
state but didn't fix this test.

Fixes: http://tracker.ceph.com/issues/20431
Signed-off-by: Sage Weil <sage@redhat.com>

cmake: compile libzfs backend conditionally

* do not REQUIRE libzfs if it is enabled, this follows the way how we
  handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
  little bit.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15874 from elliswuuuuuuuu/remove_unuse_config

common/config_opt: remove unused config

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15864 from joscollin/wip-CID1412839-uninitialized-pointer-field

common: initialize array in struct BackTrace

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15881 from joscollin/wip-1412776-explicit-null-dereferenced

common: Passing null pointer option_name to operator << in md_config_t::parse_option()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15896 from Yan-waller/wip-waller-mgrcleanup0624

mgr: drop repeated log info. and unnecessary write permission

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15910 from wido/cest-api-fork

ceph-rest-api: Various REST API fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15915 from liupan1111/wip-add-perfs

src/OSD: add more useful perf counters for performance tuning.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15917 from xiexingguo/wip-upmap-fixes

mon/OSDMonitor: a couple of upmap and other fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15935 from zphj1987/master

ceph-conf: fix typo in usage: 'mon add' should be 'mon addr'

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15885 from liewegas/wip-radosbench-timeout

qa/tasks/radosbench: increase timeout

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #15882 from rzarzynski/wip-20372

rgw: fixes for AWSBrowserUploadAbstractor auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15931 from cbodley/wip-rgw-slo-compression

rgw: use uncompressed size for range_to_ofs() in slo/dlo

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15928 from liewegas/wip-rgw-warning

rgw/rgw_op: fix whitespace and indentation warning

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15920 from wido/docs-mgr-dashboard

docs: mgr dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <jspray@redhat.com>

msg/async/rdma: avoid adding worker twice to pending_workers

Change-Id: I4dda9040a055bdb8787ffd23a6928925d0a69d83
Signed-off-by: Adir Lev <adirl@mellanox.com>

msg/async/rdma: avoid adding csi twice to pending_list

Change-Id: Id76ffbfebc54c2da339e0e7f063c5420042fbe7c
Signed-off-by: Adir lev <adirl@mellanox.com>

doc: Mgr Dashboard docs update

More instructions on how to deploy the dashboard using ceph-mgr

Signed-off-by: Wido den Hollander <wido@42on.com>

tests: rgw.py: put client roles in a separate list

Fixes: http://tracker.ceph.com/issues/20417
Signed-off-by: Nathan Cutler <ncutler@suse.com>

mon: clean up in ceph_mon.cc

The output prints to the ceph-mon.log,
so need replace cout with dout,in addition, keep cout in the usage().

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #15919 from dillaman/wip-test-cls-rbd

test: fix compile warning in ceph_test_cls_rbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15934 from myoungwon/omw-mailmap

mailmap: add Myoungwon Oh's mailmap and affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #15914 from wjwithagen/remotes/origin/wip-wjw-freebsd-osd-scrub-repair-3

test/osd/osd-scrub-repair.sh: Fix diff options on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15856 from cbodley/wip-rgw-vstart-s3tests

vstart: remove rgw_enable_static_website

Merge pull request #15863 from gmayyyha/paxosservice-use-func

mon/PaxosService: use __func__ instead of hard code function name

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15759 from ovh/bp-optimized-bs-key-escape

os/bluestore: optimized (encode|decode)_escaped

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15765 from jimmyway/wip-ec-plan-cleanup

osd/ECTransaction: cleanup the redundant check which works in overwrite IO context

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-conf: fix typo in usage: 'mon add' should be 'mon addr'

src/test/cli/ceph-conf/help.t
src/tools/ceph_conf.cc

fix add to addr

Signed-off-by: Peng Zhang zphj1987@gmail.com

mailmap: add Myoungwon Oh's mailmap and affiliation

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

rgw: use uncompressed size for range_to_ofs() in slo/dlo

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rgw_op: fix some more whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

rgw/rgw_op: fix indentation warning

/home/sage/src/ceph2/src/rgw/rgw_op.cc: In member function ‘virtual void RGWDeleteObjTags::execute()’:
/home/sage/src/ceph2/src/rgw/rgw_op.cc:777:3: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
   if (s->object.empty())
   ^~
/home/sage/src/ceph2/src/rgw/rgw_op.cc:780:5: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’
     rgw_obj obj;
     ^~~~~~~

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/radosbench: increase timeout

The current timeout isn't enough in some cases (powercycle thrashing leaves
osds down for a long time because rebooting is so slow).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14997 from ivancich/wip-bring-in-dmclock-p2

osd/PG: Add two new mClock implementations of the PG sharded operator queue

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15569 from aclamk/compress_crash_bug_refactor

rgw: Compress crash bug refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15859 from smithfarm/wip-swift-task-move-master

tests: move swift.py task from teuthology to ceph, phase one (master)

Merge pull request #15555 from liewegas/wip-19023

osd/PG: set clean when last_epoch_clean is updated

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15862 from yuyuyu101/wip-omap-enoent

kv/RocksDBStore: abort if rocksdb EIO, don't return incorrect result

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15866 from tchaikov/wip-set-secrets

auth/RotatingKeyRing: use std::move() to set secrets

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15905 from tchaikov/wip-kill-clang-warnings

test, ceph-osdomap-tool: kill clang warnings

Reviewed-by: Sage Weil <sage@redhat.com>

src/OSD: add more useful perf counters for performance tuning.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #13753 from theanalyst/wip/f/rgw/tagging

rgw: implement get/put object tags for S3

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15889 from joscollin/wip-1412614-uninitialized-scalar-field

tools/rbd_mirror: initialize non-static class member m_do_resync in ImageReplayer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: fix compile warning in ceph_test_cls_rbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon/OSDMonitor: add use_gmt_hitset to "osd pool get" cmd filter

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: drop debug_fake_ec_pool from "osd pool set" cmd filter

It is dead!

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: fix wrong sanity check order for allow_ec_overwrites

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: make blacklist expire configurable; fix potential races

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: fix wrong comment, wrong indention, wrong output etc.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: strict comparison condition

To match the specifications of straw_calc_version.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/osd_type: map.find() -> map.count() for existense checking

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: kill dead code

This option is now filtered out automatically, so these codes are unaccessable.

./bin/ceph osd unset sortbitwise
Invalid command:  sortbitwise not in full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent
osd unset full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent :  unset <key>
Error EINVAL: invalid command

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: make pg-up-items result a bit nicer

~#: ./bin/ceph osd pg-upmap-items 0.7 0 2 1 3 4 5
set 0.7 pg_upmap_items mapping to [0->2,1->3,4->5]

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: reject updating upmap if number of osd overflows/underflows

This can avoid, e.g.:

~#:./bin/ceph osd pg-upmap 0.7 1 2 3 4
set 0.7 pg_upmap mapping to [1,2,3,4]

~#:./bin/ceph pg dump
PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE        STATE_STAMP                VERSION REPORTED UP        UP_PRIMARY ACTING    ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP                LAST_DEEP_SCRUB DEEP_SCRUB_STAMP
0.7           0                  0        0         0       0     0   0        0      peering 2017-06-24 10:11:03.938102     0'0    53:18 [1,2,3,4]          1 [1,2,3,4]              1        0'0 2017-06-24 09:38:50.162197             0'0 2017-06-24 09:38:50.162197

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: make apply_upmap handle multiple pg-upmap-items correctly

Was:
PG_STAT  UP      UP_PRIMARY ACTING  ACTING_PRIMARY
0.7      [1,0,4]          1 [1,0,4]              1

~#:./bin/ceph osd pg-upmap-items 0.7 0 2 1 3 4 5
set 0.7 pg_upmap_items mapping to [0,2,1,3,4,5]

PG_STAT  UP      UP_PRIMARY ACTING  ACTING_PRIMARY
0.7      [1,2,4]          1 [1,2,4]              1

Now:
PG_STAT  UP      UP_PRIMARY ACTING  ACTING_PRIMARY
0.7      [1,0,4]          1 [1,0,4]              1

~#:./bin/ceph osd pg-upmap-items 0.7 0 2 1 3 4 5
set 0.7 pg_upmap_items mapping to [0,2,1,3,4,5]

PG_STAT  UP      UP_PRIMARY ACTING  ACTING_PRIMARY
0.7      [3,2,5]          3 [3,2,5]              3

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test/osd/osd-scrub-repair.sh: Fix diff options

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

rest api: Set application variable when not executed as __main__

This allows for using this Python file as a WSGI module as it expects
a variable called application.

Signed-off-by: Wido den Hollander <wido@42on.com>

rest api: Set default ceph.conf and id options

This allows for easy execution of the REST API:

  $ /usr/bin/ceph-rest-api

Signed-off-by: Wido den Hollander <wido@42on.com>

rest api: Listen on :: (IPv6) instead of 0.0.0.0

On Linux systems a socket listening on :: also listens on
all IPv4 addresses.

This makes sure the API is usable over IPv6 as well.

Signed-off-by: Wido den Hollander <wido@42on.com>

common: Passing null pointer option_name to operator << in md_config_t::parse_option()

Fixes the Coverity Scan Report:
CID 1412776 (#1 of 1): Explicit null dereferenced (FORWARD_NULL)19. var_deref_model: Passing null pointer option_name to operator <<, which dereferences it.

Fixed the review comments too in this commit.

Signed-off-by: Jos Collin <jcollin@redhat.com>

radosgw: refactor compression code

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

radosgw, compression: added unit test

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

rgw: connection reset/crashed when download large zero object with compression enable

large zero object has a large compression rate,
even 4M compressed data can decompress several GB data.
Handle so much data in single process lead strange issue.

Fixes: http://tracker.ceph.com/issues/20098
Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com

os/ZFS: make it compilable

* `cct` is not globally avaiable, use cct() wrapper instead
* libzfs.h is always avaiable here. so remove the guard

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15445 from wjwithagen/wip-wjw-freebsd-osd-scrub-repair-2

test/osd/osd-scrub-repair.sh: disable ec_overwrite tests on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: drop unnecessary write permission

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

mgr: use readable errno strings

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #15902 from joscollin/wip-1411820-Division-or-modulo-by-zero

test: Division by zero in Legacy::encode_n()

Reviewed By: Brad Hubbard <bhubbard@redhat.com>

ceph-osdomap-tool: early return and remove never-read store statement

* silence clang analyzer's warning of: "Value stored to 'r' is never
  read"
* replace the "goto" statement with early return to improve the
  readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15903 from Adirl/data_path_fixes

msg/async/rdma: Data path fixes

Reviewed-by: Haomai Wang <haomai@xsky.com>

test/test_async_driver: silence clang static analyzer warning

* silence the warning of: Value stored to 'r' is never read
* update the gtest assertions to be semantically more correct.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: swift.py: tweak imports

The ".." form only works within the teuthology repo. With swift.py now in the
Ceph repo, we have to be explicit.

Error message was: "ValueError: Attempted relative import beyond toplevel
package

Signed-off-by: Nathan Cutler <ncutler@suse.com>

test: Division by zero in Legacy::encode_n()

Fixes the Coverity Scan Report:
CID 1411820 (#1 of 1): Division or modulo by zero (DIVIDE_BY_ZERO)
9. divide_by_zero: In expression bl.length() * i / sum, division by expression sum which may be zero has undefined behavior.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15826 from xiexingguo/wip-bluestore-20170622

os/bluestore: cleanup min_alloc_size; some formatting nits

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15865 from tchaikov/wip-mgr-pg-command

mgr: enable ceph_send_command() to send pg command

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15875 from xiexingguo/wip-osdmon-20170623

mon/OSDMonitor: "osd crush class rename" support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15890 from tchaikov/wip-19741

osdc/Objecter: release message if it's not handled

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15891 from minggr/wip-sdata_op_ordering_lock

osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid miss…

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15899 from theanalyst/doc/release-typo

doc: Luminous release notes typo fixes

Merge pull request #15901 from joscollin/wip-CID1412577-Division-or-modulo-by-float-zero

mon: Division by zero in PGMapDigest::dump_pool_stats_full()

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15900 from smithfarm/wip-docdir-fix

build/ops: rpm: sane packaging of %{_docdir}/ceph directory

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: allocated reged mem can be greater than requires len

Change-Id: I3ec42afca39c2ad93fc66ed69b3609b68f315e34
Signed-off-by: Adir lev <adirl@mellanox.com>

msg/async/rdma: if failed to send push csi to the back

Change-Id: If6c72917b6e3a836279b86a4d5e9e54383d7b4bd
Signed-off-by: Adir lev <adirl@mellanox.com>

mon: Division by zero in PGMapDigest::dump_pool_stats_full()

Fixes The Coverity Scan Report:
CID 1412577 (#1 of 1): Division or modulo by float zero (DIVIDE_BY_ZERO)
35. divide_by_zero: In expression (float)mk / k, division by expression k which may be zero has undefined behavior.

Signed-off-by: Jos Collin <jcollin@redhat.com>