doc: multimds is no longer experimental

Signed-off-by: John Spray <john.spray@redhat.com>

qa: update cephtool test for multimds on by default

Signed-off-by: John Spray <john.spray@redhat.com>

qa: no longer need to explicitly enable multimds

Signed-off-by: John Spray <john.spray@redhat.com>

mds: enable multimds by default in new filesystems

Signed-off-by: John Spray <john.spray@redhat.com>

mon: remove experimental warning on multimds

Signed-off-by: John Spray <john.spray@redhat.com>

mds: don't call StrayManager::eval_stray() for undefined inode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: drop locks before waiting for export targets

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: handle MDirUpdate race

mds may try discover several times for MDirUpdate, rename may kick
in and cause MDCache::path_traverse() to return error.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: don't forge replica dirfrag

MDCache::forge_replica_dir() set wrong dir_auth if the forged replica
dirfrag is subtree root.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: avoid submitting log entry while adjusting subtree map

MDCache::eval_subtree_root() may tigger scatter-gather process, which
submits log entry. Submitting log entry while adjusting subtree map is
bad, because subtree map in intermediate state may get used/logged.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: don't mark nestlock dirty on improper inode

If inode is replica and it has no auth subtree dirfrag, We should
not mark its nestlock dirty.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: create subtree root immediately after directory tree becomes frozen

When a directory tree become frozen, its WAIT_FROZEN contexts are
executed asynchronously. Before Migrator::export_frozen() set export
bounds, MDCache::try_subtree_merge_at() can merge newly imported
subtree into the frozen directory tree. This causes problem if there
are auth pins in newly imported subtree.

The fix is creating subtree root immediately after directory tree
becomes frozen. The new subtree root has dir_auth 'me, me', so it's
not meregeable.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: fix stray dentry replication in cache rejoin ack

To replicate s stray dentry, we need to replicate all its ancestors.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #15795 from myoungwon/wip-print-ignore_redirect

osd/osd_types: add flag name (IGNORE_REDIRECT)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15656 from aclamk/download_err_with_comp_followup

rgw: use 64-bit offsets for compression

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15848 from xiexingguo/wip-fix-rmcc

src/vstart.sh: kill dead upmap option

Merge pull request #15851 from liewegas/wip-luminous-notes

doc/release-notes: update luminous notes

doc/release-notes: 'osd crush class rename' is coming

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: ceph tell <foo> help

Signed-off-by: Sage Weil <sage@redhat.com>

doc/start/os-recommendations: update

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: note debian stretch addition

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: sleep settings

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: link to EC docs

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: update RGW metadata

Signed-off-by: Sage Weil <sage@redhat.com>

dev/release-notes: various updates from other PR

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: notes on new CLI commands

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'mon feature list' -> 'mon feature ls'

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: update luminous notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15764 from tchaikov/wip-20342

qa/suites/upgrade/hammer-jewel-x: add luminous.yaml

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15877 from wjwithagen/wip-wjw-vstart-ceph-mgr-restfull

vstart.sh: Work around mgr restfull not available

./src/vstart.sh: Work around mgr restfull not available

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #15832 from Adirl/pd_assert

msg/async/rdma: remove assert from ibv_dealloc_pd in ProtectionDomain

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15283 from donglinpeng/copyPartUpload

rgw: multipart copy-part remove '/' for s3 java sdk request header.
(make check failures not related)

Merge pull request #15857 from theanalyst/doc/v12.1.0/release-notes-3

doc: v12.1.0 release notes notable changes addition again

doc: v12.1.0 release notes notable changes addition again

commmits from 9f259cb923e7474229bb2e5e54b59f3b7f068bc5..luminous && sort

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #15819 from linuxbox2/wip-clru-lockfix

rgw_file: avoid a recursive lane lock in LRU drain
(failing make check is unrelated)

rgw_file: avoid a recursive lane lock in LRU drain

While highly unlikely in normal fs operations, draining an
entire filesystem could induce a recursion on object delete/unref,
when provoked by cohort_lru::drain().

Adjusted to use intrusive slist rather than std:vector<T*>
after review.

Found running librgw_file_nfsns unit test.

Fixes: http://tracker.ceph.com/issues/20374
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

radosgw: fixed sizes of integers used to operate on offsets and lengths in compression,
         followup of http://tracker.ceph.com/issues/20231

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Merge pull request #15814 from liewegas/wip-everyone-hates-btrfs

qa/objectstore/filestore-btrfs: test btrfs on trusty only

src/vstart.sh: kill dead upmap option

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15820 from liewegas/wip-20303

mon/PGMap: slightly better debugging around pgmap updates

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15846 from xiexingguo/wip-remap-upmap

mon/OSDMonitor: _apply_remap -> _apply_upmap; less code redundancy

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: less code redundancy

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15805 from jcsp/wip-mds-flush-upgrade

client: check for luminous MDS before sending FLUSH_MDLOG

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15800 from ukernel/wip-20340

mds: save  projected path into inode_t::stray_prior_path

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15776 from ukernel/wip-20338

osdc/Journaler: fix memory leak in Journaler::_issue_read()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15430 from jcsp/wip-20165

mds: fix hangs involving re-entrant calls to journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15548 from ukernel/wip-20196

mds: improvements for stray reintegration

Reviewed-by: John Spray <john.spray@redhat.com>

mon/MonCommand: fix description of "osd pg-upmap"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: _apply_remap -> _apply_upmap

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15796 from lixiaoy1/fio_ceph_objectstore

fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled

Merge pull request #15753 from pritha-srivastava/wip-rgw-s3tests-conf

rgw: Changes for s3test config file, to add user under a tenant.
Reviewed-by: Casey Bodely <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

msg/async/rdma: remove assert from ibv_dealloc_pd in ProtectionDomain
destructor

issue: 1044307

Change-Id: I2fe2099c5f4403f14877e0fbe5d4868dd34d4f9b
Signed-off-by: DanielBar-On <danielbo@mellanox.com>

Merge pull request #15317 from liupan1111/wip-fix-list

rbd-nbd: display pool/image/snap information in list output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15803 from mikulely/cleanup-rgw_rados_doc

rgw: update function doc in rgw_rados.h  and rgw_rados.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>

qa/workunits/rbd: do not test --export-format unless supported

in 55edd81, test for `--export-format` was added to exercise this
option. but this option is only supported on luminous, so we need to
check if it's avaialble before using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

a/workunits/rados/test_envlibrados_for_rocksdb.sh: git clone --depth 1

no need to clone the whole history of rocksdb, we just need the HEAD of
master. so "--depth 1" is better and faster in this case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade/hammer-jewel-x: upgrade all mon to luminous before osd

luminous osd requires that monmap has REQUIRE_LUMINOUS before it boots.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade/hammer-jewel-x: replace kraken.yaml with luminous.yaml

* add mgr.x to roles
* to setup mgr and set the require-osd-release bit in osdmap
* do not restart an osd for waiting for healthy: the cluster is not
  healthy until the require-osd-release=luminous is set in osdmap, and a
  mgr is up and running.

Fixes: http://tracker.ceph.com/issues/20342
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15261 from mychoxin/yuanxin_use_func

os/filestore: when print log, use __func__ instead of hard code function name

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15354 from badone/wip-rados-ls-auth-fix

osd: Reverse order of op_has_sufficient_caps and do_pg_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15649 from liewegas/wip-20274

osd/PrimaryLogPG: record prior_version for DELETE events

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15601 from liewegas/wip-no-restful-cert

mgr: remove default cert; disable both restful and dashboard by default

Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: update rgw_rados.h function doc

+ update system_obj_get_attr func comment
+ drop deprecated func comment for get_obj
+ update func comment for pool_iterate_begin

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #15755 from Adirl/wip-HUGEPAGES

msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf

Reviewed-by: Haomai Wang <haomai@xsky.com>

qa/workunits/rest/test_mgr_rest_api: remove ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/rest/mgr-restful: configure cert

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: add 'restful restart' command

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15757 from runsisi/wip-image-ts

librbd: add create timestamp metadata for image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15444 from wjwithagen/wip-wjw-freebsd-termwidth

test: Make screencandy optional for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PGLog: ignore ERROR entires in _merge_object_divergent_entries

Sometimes ERROR log entries do not have prior_version.  Also,
they aren't actually updates and don't affect the object.

Fixes: http://tracker.ceph.com/issues/20274
Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: slightly better debugging around pgmap updates

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15783 from liewegas/wip-20358

os/bluestore: move sharedblob to new collection in same shard

Merge pull request #15785 from liewegas/wip-bluestore-fsck

qa/objectstore/bluestore*: fsck on mount

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/objectstore/filestore-btrfs: test btrfs on trusty only

Centos btrfs hits ENOSPC all the time :(, and now xenial is doing
it too.

See http://tracker.ceph.com/issues/20169
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15734 from dillaman/wip-librbd-image-remove

librbd: fix issues with image removal state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15727 from ovh/bp-runtime-cfg-of-hb-inflator

osd, messages/MOSDPing: bunch of fixes related to ping inflation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15723 from badone/wip-ceph-mon-startup-error-logging

mon: Log errors at startup

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15784 from liewegas/wip-thrash-cleanup

qa/tasks/ceph_manager: get osds all in after thrashing

Merge pull request #15787 from liewegas/wip-19943-fix

ceph-objectstore-tool: do not populate snapmapper with missing clones

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15813 from tchaikov/wip-cmake

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

pybind/mgr/restful: remove default path names for the cert

I don't think these are useful.

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: do not start if no certificate is configured

This removes the default filename, by the way.  We also work around a
problem with make_server where it sets up the socket to listen before
checking for the cert, thereby making it problematic to rebind to the
port shortly thereafter when we do have a socket.  (SO_REUSEADDR would
be appropriate but there doesn't seem to be an easy way to make
make_server use it.)

Signed-off-by: Sage Weil <sage@redhat.com>

restful: Introduce create-self-signed-cert command

This implements a create-self-signed-cert command for the ceph-mgr
restful interface.

It is designed so that it will try to restart the module once the cert
is created.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #15060 from shashalu/fix-slo-dlo-range

rgw: fix SLO/DLO range requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15791 from liewegas/wip-20360

Revert "qa: do not restrict valgrind runs to centos"

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13683 from liewegas/wip-crush-rulesets

crush: detect and (usually) fix ruleset != rule id

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15789 from linuxbox2/wip-nfs-doc

RGW NFS: add nfs.rst to doc/radosgw
(override because make check fail [missing selinux policy help file] unrelated)

RGW NFS: add nfs.rst to doc/radosgw

This document contains sections by John Wilkins <jowilkin@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15255 from david-z/wip-mgr-fsstatus

pybind/mgr/fsstatus: use mds_mem.dn as dentry counter

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15621 from mingshuaiwang/master

mds: mds perf item 'l_mdl_expos' always behind journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>

fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled

Currently users can't use ceph admin command to get db stats when running
bluestore fio plugin. This patch is to print db stats when rocksdb_perf
is set in config file.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Merge pull request #15662 from renhwztetecs/renhw-wip-client-trim_cache

client: call the lru_remove() twice,when trim cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15715 from jcsp/wip-jt-rmapply

tools/cephfs: remove `apply` mode of cephfs-journal-tool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15779 from renhwztetecs/renhw-wip-client-dentry

client: fix Dentry::dump

Reviewed-by: John Spray <john.spray@redhat.com>

client: check for luminous MDS before sending FLUSH_MDLOG

This is needed to make it safe to run newer clients
on pre-luminous MDS.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: save  projected path into inode_t::stray_prior_path

Otherwise, path string like #10000000000/xxx may get saved into
inode_t::stray_prior_path.

Fixes: http://tracker.ceph.com/issues/20340
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

osd/osd_types: add flag name (IGNORE_REDIRECT)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

rgw: fix SLO/DLO range requests

Sending any SLO/DLO range requests will retrun InvalidRange error. SLO/DLO should call range_to_ofs after got object size instead of read_op.prepare() to deal with negative offsets.

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>