doc/start/os-recommendations: update

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: note debian stretch addition

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: sleep settings

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: link to EC docs

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: update RGW metadata

Signed-off-by: Sage Weil <sage@redhat.com>

dev/release-notes: various updates from other PR

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: notes on new CLI commands

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'mon feature list' -> 'mon feature ls'

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: update luminous notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15764 from tchaikov/wip-20342

qa/suites/upgrade/hammer-jewel-x: add luminous.yaml

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15877 from wjwithagen/wip-wjw-vstart-ceph-mgr-restfull

vstart.sh: Work around mgr restfull not available

./src/vstart.sh: Work around mgr restfull not available

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #15832 from Adirl/pd_assert

msg/async/rdma: remove assert from ibv_dealloc_pd in ProtectionDomain

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15283 from donglinpeng/copyPartUpload

rgw: multipart copy-part remove '/' for s3 java sdk request header.
(make check failures not related)

Merge pull request #15857 from theanalyst/doc/v12.1.0/release-notes-3

doc: v12.1.0 release notes notable changes addition again

doc: v12.1.0 release notes notable changes addition again

commmits from 9f259cb923e7474229bb2e5e54b59f3b7f068bc5..luminous && sort

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #15819 from linuxbox2/wip-clru-lockfix

rgw_file: avoid a recursive lane lock in LRU drain
(failing make check is unrelated)

rgw_file: avoid a recursive lane lock in LRU drain

While highly unlikely in normal fs operations, draining an
entire filesystem could induce a recursion on object delete/unref,
when provoked by cohort_lru::drain().

Adjusted to use intrusive slist rather than std:vector<T*>
after review.

Found running librgw_file_nfsns unit test.

Fixes: http://tracker.ceph.com/issues/20374
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15814 from liewegas/wip-everyone-hates-btrfs

qa/objectstore/filestore-btrfs: test btrfs on trusty only

Merge pull request #15820 from liewegas/wip-20303

mon/PGMap: slightly better debugging around pgmap updates

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15846 from xiexingguo/wip-remap-upmap

mon/OSDMonitor: _apply_remap -> _apply_upmap; less code redundancy

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: less code redundancy

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15805 from jcsp/wip-mds-flush-upgrade

client: check for luminous MDS before sending FLUSH_MDLOG

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15800 from ukernel/wip-20340

mds: save  projected path into inode_t::stray_prior_path

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15776 from ukernel/wip-20338

osdc/Journaler: fix memory leak in Journaler::_issue_read()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15430 from jcsp/wip-20165

mds: fix hangs involving re-entrant calls to journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15548 from ukernel/wip-20196

mds: improvements for stray reintegration

Reviewed-by: John Spray <john.spray@redhat.com>

mon/MonCommand: fix description of "osd pg-upmap"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: _apply_remap -> _apply_upmap

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15796 from lixiaoy1/fio_ceph_objectstore

fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled

Merge pull request #15753 from pritha-srivastava/wip-rgw-s3tests-conf

rgw: Changes for s3test config file, to add user under a tenant.
Reviewed-by: Casey Bodely <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

msg/async/rdma: remove assert from ibv_dealloc_pd in ProtectionDomain
destructor

issue: 1044307

Change-Id: I2fe2099c5f4403f14877e0fbe5d4868dd34d4f9b
Signed-off-by: DanielBar-On <danielbo@mellanox.com>

Merge pull request #15317 from liupan1111/wip-fix-list

rbd-nbd: display pool/image/snap information in list output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15803 from mikulely/cleanup-rgw_rados_doc

rgw: update function doc in rgw_rados.h  and rgw_rados.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>

qa/workunits/rbd: do not test --export-format unless supported

in 55edd81, test for `--export-format` was added to exercise this
option. but this option is only supported on luminous, so we need to
check if it's avaialble before using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

a/workunits/rados/test_envlibrados_for_rocksdb.sh: git clone --depth 1

no need to clone the whole history of rocksdb, we just need the HEAD of
master. so "--depth 1" is better and faster in this case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade/hammer-jewel-x: upgrade all mon to luminous before osd

luminous osd requires that monmap has REQUIRE_LUMINOUS before it boots.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade/hammer-jewel-x: replace kraken.yaml with luminous.yaml

* add mgr.x to roles
* to setup mgr and set the require-osd-release bit in osdmap
* do not restart an osd for waiting for healthy: the cluster is not
  healthy until the require-osd-release=luminous is set in osdmap, and a
  mgr is up and running.

Fixes: http://tracker.ceph.com/issues/20342
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15261 from mychoxin/yuanxin_use_func

os/filestore: when print log, use __func__ instead of hard code function name

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15354 from badone/wip-rados-ls-auth-fix

osd: Reverse order of op_has_sufficient_caps and do_pg_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15649 from liewegas/wip-20274

osd/PrimaryLogPG: record prior_version for DELETE events

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15601 from liewegas/wip-no-restful-cert

mgr: remove default cert; disable both restful and dashboard by default

Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: update rgw_rados.h function doc

+ update system_obj_get_attr func comment
+ drop deprecated func comment for get_obj
+ update func comment for pool_iterate_begin

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #15755 from Adirl/wip-HUGEPAGES

msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf

Reviewed-by: Haomai Wang <haomai@xsky.com>

qa/workunits/rest/test_mgr_rest_api: remove ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/rest/mgr-restful: configure cert

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: add 'restful restart' command

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15757 from runsisi/wip-image-ts

librbd: add create timestamp metadata for image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15444 from wjwithagen/wip-wjw-freebsd-termwidth

test: Make screencandy optional for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PGLog: ignore ERROR entires in _merge_object_divergent_entries

Sometimes ERROR log entries do not have prior_version.  Also,
they aren't actually updates and don't affect the object.

Fixes: http://tracker.ceph.com/issues/20274
Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: slightly better debugging around pgmap updates

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15783 from liewegas/wip-20358

os/bluestore: move sharedblob to new collection in same shard

Merge pull request #15785 from liewegas/wip-bluestore-fsck

qa/objectstore/bluestore*: fsck on mount

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/objectstore/filestore-btrfs: test btrfs on trusty only

Centos btrfs hits ENOSPC all the time :(, and now xenial is doing
it too.

See http://tracker.ceph.com/issues/20169
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15734 from dillaman/wip-librbd-image-remove

librbd: fix issues with image removal state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15727 from ovh/bp-runtime-cfg-of-hb-inflator

osd, messages/MOSDPing: bunch of fixes related to ping inflation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15723 from badone/wip-ceph-mon-startup-error-logging

mon: Log errors at startup

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15784 from liewegas/wip-thrash-cleanup

qa/tasks/ceph_manager: get osds all in after thrashing

Merge pull request #15787 from liewegas/wip-19943-fix

ceph-objectstore-tool: do not populate snapmapper with missing clones

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15813 from tchaikov/wip-cmake

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

pybind/mgr/restful: remove default path names for the cert

I don't think these are useful.

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: do not start if no certificate is configured

This removes the default filename, by the way.  We also work around a
problem with make_server where it sets up the socket to listen before
checking for the cert, thereby making it problematic to rebind to the
port shortly thereafter when we do have a socket.  (SO_REUSEADDR would
be appropriate but there doesn't seem to be an easy way to make
make_server use it.)

Signed-off-by: Sage Weil <sage@redhat.com>

restful: Introduce create-self-signed-cert command

This implements a create-self-signed-cert command for the ceph-mgr
restful interface.

It is designed so that it will try to restart the module once the cert
is created.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #15060 from shashalu/fix-slo-dlo-range

rgw: fix SLO/DLO range requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15791 from liewegas/wip-20360

Revert "qa: do not restrict valgrind runs to centos"

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13683 from liewegas/wip-crush-rulesets

crush: detect and (usually) fix ruleset != rule id

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15789 from linuxbox2/wip-nfs-doc

RGW NFS: add nfs.rst to doc/radosgw
(override because make check fail [missing selinux policy help file] unrelated)

RGW NFS: add nfs.rst to doc/radosgw

This document contains sections by John Wilkins <jowilkin@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15255 from david-z/wip-mgr-fsstatus

pybind/mgr/fsstatus: use mds_mem.dn as dentry counter

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15621 from mingshuaiwang/master

mds: mds perf item 'l_mdl_expos' always behind journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>

fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled

Currently users can't use ceph admin command to get db stats when running
bluestore fio plugin. This patch is to print db stats when rocksdb_perf
is set in config file.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Merge pull request #15662 from renhwztetecs/renhw-wip-client-trim_cache

client: call the lru_remove() twice,when trim cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15715 from jcsp/wip-jt-rmapply

tools/cephfs: remove `apply` mode of cephfs-journal-tool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15779 from renhwztetecs/renhw-wip-client-dentry

client: fix Dentry::dump

Reviewed-by: John Spray <john.spray@redhat.com>

client: check for luminous MDS before sending FLUSH_MDLOG

This is needed to make it safe to run newer clients
on pre-luminous MDS.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: save  projected path into inode_t::stray_prior_path

Otherwise, path string like #10000000000/xxx may get saved into
inode_t::stray_prior_path.

Fixes: http://tracker.ceph.com/issues/20340
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

rgw: fix SLO/DLO range requests

Sending any SLO/DLO range requests will retrun InvalidRange error. SLO/DLO should call range_to_ofs after got object size instead of read_op.prepare() to deal with negative offsets.

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

Merge pull request #15728 from majianpeng/bluefs-remove-memcopy-for-continue-memory

os/bluestore/BlueFS: Rebuild memcopy for bufferlist::page_aligned_app…

Reviewed-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: test page_aligned_appender

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: replace claim_append with claim_append_piecewise to reduce memory copy.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/buffer: add function  bufferlist::claim_append_piecewise(list&  bl).

This only useful for bl is bufferlist::page_aligned_appender. Using
this function can remove memcopy for continue ptrs.
Because page_aligned_appender::flush will split a ptr into two or
more ptrs. For this case, rebuild_aligned_size_and_memory can't handle,
it  will rebuild.

For example
a=bl.get_page_aligned_appender(1);
a.append(3K)
a.flush();
t.claim_append(bl);
a.append(1K);
a.flush();
t.claim_append(bl);

dst.claim_append(t);
//3K and 1K ptr are continue in memory. But they are two ptrs..
dst.is_aligned_size_and_memory(4096,4096) is false.

We add new function claim_append_piecewise() to specially
handle this case.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: replicated_ruleset -> replicated_rule

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: remove default of 127.0.0.1

i.e., disabled until explicitly enabled by an admin

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/dashboard: remove default of 127.0.0.1

i.e., disabled until explicitly enabled by an admin.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec,debian: remove auto-generated self-signed mgr restful cert

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "qa: do not restrict valgrind runs to centos"

This reverts commit 5923961465e1efd3cc6bc072fef293dda3d975d0.

See http://tracker.ceph.com/issues/20360

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15260 from shashalu/fix-secondary-zonegroup-cors

rgw: update bucket cors in secondary zonegroup should forward to master

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15709 from jcsp/wip-dashboard-loglast

mgr/dashboard: load log lines on startup, split out audit log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15750 from xiexingguo/wip-bluestore-20170619

os/bluestore: refactor BlueStore::_do_write;  kill dead ExtentMap::find_lextent() method

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15766 from liewegas/wip-20227-fix

os/bluestore: fix unsharing blob dirty_range args

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15014 from zhangsw/fix-rgw-versionIdMarker-bug

rgw: VersionIdMarker and NextVersionIdMarker should be returned when listing object versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14500 from zhangsw/fix-rgw-multipart-index-bug

rgw: fix the bug that part's index can't be removed after completing multipart upload

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

os/bluestore: debug OmapIteratorImpl

Signed-off-by: Sage Weil <sage@redhat.com>

osd/SnapMapper: debug keys inserted, removed, and iterated

Hunting bug http://tracker.ceph.com/issues/19943

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: do not populate snapmapper with missing clones

Cache pools may have clones represented in SnapSet that are not stored
locally.  This is fine as long as ceph-objectstore-tool doesn't go and
add them when it shouldn't.

Fixes: http://tracker.ceph.com/issues/19943
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15771 from fangyuxiangGL/cleanup_rgw_admin

rgw: cleanup unused codes in rgw_admin.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15768 from ukernel/wip-objecter-inflight-ops

osdc/Objecter: fix inflight_ops update

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15737 from wjwithagen/wip-wjw-test_rgw_string

rgw/rgw_string.h: FreeBSD would like errno.h included

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15782 from cbodley/wip-rgw-vstart-s3tests

vstart: add rgw configuration needed to pass all s3tests

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>