qa: update handling of fs status format

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

PendingReleaseNotes: add note for format change

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds/MDSMap : use arrary_section for mds stat

http://tracker.ceph.com/issues/22338
Signed-off-by: You Ji <youji@ebay.com>

Merge pull request #19188 from adamwg/awg/rbd-watchers-api

librbd: Add a function to list image watchers

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19337 from trociny/wip-22306

pybind/rbd: fix metadata functions error handling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18967 from shun-s/wip-rbd-cp-optimize

librbd: speed up sparse copy when object map is available

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19251 from MahatiC/eventtrace_librbd

librbd: add eventtrace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19280 from Songweibin/wip-parent-info

rbd: add parent info when moving child into trash bin

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19315 from trociny/wip-fsx-ggate

test/librbd: rbd-ggate mode for fsx

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #18791 into master

* refs/pull/18791/head:
mds: bump protocol for snapshot changes
mds: force client to do snapflush
mds: check existance of snaps when rebuilding need_snapflush
mds: properly setup need_snapflush for flushsnap messages
mds: properly update CInode::first and CDentry first
mds: don't create snap inode during trans-authority rename
mds: trim non-auth snap dentry during cache rejoin
mds: detach unlinked inode's snaprealm from its parent
mds: fix CInode::find_snaprealm()
mds: fix snap dentry discover
mds: introduce MDSMap::get_mds_set_lower_bound()
mds: properly distinguish snaptable TABLE_OP_CREATE from TABLE_OP_UPDATE
mds: properly set reply buffer for table request 'agree' message
mds: make change to mds table after corresponding event get logged

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19351 from ceph/wip-rm22281

ceph-volume rollback on failed OSD prepare/create

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19344 from tchaikov/wip-we-need-cxx11-and-more

cmake: bail out if GCC version is less than 5.1

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19343 from dillaman/wip-22321

ceph-dencoder: moved RBD types outside of RGW preprocessor guard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19333 from amitkumar50/cov-osd-4

osd: Initializing start_offset,last_offset,offset

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #19334 from amitkumar50/cov-misc-13

test: Initializing ChunkReadOp members

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #18849 into master

* refs/pull/18849/head:
mds:improve mds perf counters can statistics filesystem operations number and latency

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18941 into master

* refs/pull/18941/head:
discard the mdsload clear after prep_rebalance in case we want to export it for debugging
make sure that MDBalancer uses heartbeat info from the same epoch

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19089 into master

* refs/pull/19089/head:
client: xlist link dentries instead of set
client: cleanup Dentry definition

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19092 into master

* refs/pull/19092/head:
mds: remove useless check in Migrator::export_dir
mds: ignore export pin for unlinked directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #19218 into master

* refs/pull/19218/head:
client: quit on failed remount during dentry invalidate test

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19224 into master

* refs/pull/19224/head:
mds: use find instead of count in CInode::get_dirfrag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #19237 into master

* refs/pull/19237/head:
mds: get rid of the redundant code

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19347 from ceph/wip-rm22326

ceph-volume: warn on missing ceph.conf file

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume lvm.create rollback osd creation when prepare or activate fails

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.create fallback to bluestore when objectstore is not specified

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare fallback to bluestore when objectstore is not specified

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare rollback osd when prepare fails and an osd id was generated

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.common create a rollback_osd utility to cleanup failed osd prepare/create calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>

pybind/rbd: raise KeyError when metadata does not exist

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

ceph-volume test logging ignored ceph.conf file

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume main warn on inability to load ceph.conf, don't raise

Signed-off-by: Alfredo Deza <adeza@redhat.com>

test: Initializing ChunkReadOp members

Fixes the coverity issue:
2. uninit_member: Non-static class member offset is not
initialized in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member length is not
initialized in this constructor nor in any functions that it calls.

CID 1424433 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
6. uninit_member: Non-static class member tgt_offset is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

osd: Initializing start_offset,last_offset,offset

Fixes the coverity issues:

2. uninit_member: Non-static class member start_offset
is not initialized in this constructor nor in any functions
that it calls.

CID 1424396 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member last_offset is not
initialized in this constructor nor in any functions that it calls.

CID 1424658 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member offset is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

cmake: bail out if GCC version is less than 5.1

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dencoder: moved RBD types outside of RGW preprocessor guard

Fixes: http://tracker.ceph.com/issues/22321
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/rbd: fix metadata functions error handling

Fixes: http://tracker.ceph.com/issues/22306
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19101 from ifed01/wip-ifed-extending-fio

test/fio: extend fio objectstore plugin to better simulate OSD behavior

Merge pull request #18863 from tchaikov/wip-devtoolset-7

build/ops: use devtoolset-7 on centos/rhel-7

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18812 from tchaikov/wip-pybind-rados-service-api

pybind/rados: add rados_service_*()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19307 from tchaikov/wip-drop-ubuntu-trusty

qa,doc: drop support of ubuntu trusty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rbd: comment/msg cleanup

Signed-off-by: songweibin <song.weibin@zte.com.cn>

rbd: list parent info with "--long" when image child is in trash

We can get child's parent information even it's in
trash bin if "--long" flag is specified.

Signed-off-by: songweibin <song.weibin@zte.com.cn>

mds: bump protocol for snapshot changes

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: force client to do snapflush

Snap inodes that are waiting for flush are auth pinned, they affect
subtree/dirfrag frozen. Force client to do snapflush after waiting
for a period of time.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: check existance of snaps when rebuilding need_snapflush

do nothing if there is no snap

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: properly setup need_snapflush for flushsnap messages

When processing a cap flush message that is re-sent, it's possble
that the sender has already released all WR caps. So can't rely on
CEPH_CAP_ANY_WR check in MDCache:cow_inode().

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: properly update CInode::first and CDentry first

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

install-deps.sh: use gcc-7 on trusty

* always install gcc-7 on trusty
* point g++ to g++-7 if not yet

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec: use devtoolset-7-gcc-c++ on centos/rhel

for better performance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: always use yum-config-manager for consistency

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: install devtoolset-7 for centos/rhel

ready them for gcc7 and libstdc++-7 for better performance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19324 from neha-ojha/wip-fix-collectl

qa: install collectl with cbt task

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19325 from ceph/wip-ceph-disk-man-fix

doc/man remove deprecation of ceph-disk man page title

Reviewed-by: Brad Hubbard

client: quit on failed remount during dentry invalidate test

Fixes: http://tracker.ceph.com/issues/22269
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

client: xlist link dentries instead of set

This saves space and avoids unnecessary set logic. In particular, we no longer
need to do a heap allocation for each Dentry * in the std::set.

Before:

(gdb) print sizeof(Inode)
$1 = 1336
(gdb) print sizeof(Inode::dn_set)
$2 = 48

After:

(gdb) print sizeof(Inode)
$1 = 1360
(gdb) print sizeof(Inode::dentries)
$2 = 24

I'm not sure why the Inode size increased when the member size decreased (weird
padding by g++)? Anyway, we still get the benefit of no heap allocations for
the Dentry *s.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc/man remove deprecation of ceph-disk man page title

Signed-off-by: Alfredo Deza <adeza@redhat.com>

qa: install collectl with cbt task

Signed-off-by: Neha Ojha <nojha@redhat.com>

test/librbd: rbd-ggate mode for fsx

Right now it works only in lite mode due to ggate kernel driver
does not support resize. Also all operations should be block size
alligned, thus -r and -w options should be properly set.

Example:

  ceph_test_librbd_fsx -G -L -r 512 -w 512 -Z -d -N 1024 rbd test

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

test/librbd: enable fsx lite mode support

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #19241 from ceph/wip-doc-deprecate-cdisk

doc: update references to use ceph-volume

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19321 from ceph/wip-rm22305

ceph-volume should be able to handle multiple LVM (VG/LV) tags

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19319 from ceph/wip-rm22297

ceph-volume  handle inline comments in the ceph.conf file

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.api ensure that we can handle non-ceph tags in lvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19316 from trociny/wip-blkdev-fixup

common/blkdev: fix build in FreeBSD environment

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

ceph-volume api.lvm only consider 'ceph.' tags in logical volumes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/install use ceph-volume in manual deployment steps

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update user-management to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update bluestore-migration to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update add-or-rm-osds to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update bluestore config to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/rados update auth config examples to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/man add deprecation notices to ceph-disk man page

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19275 from qrGitHub/wip-rgw-correct-comment

rgw: correct comment in function parse_credentials

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-volume configuration allow inlined comments for # and ;

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests verify that INI comments can be inlined

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19314 from ifed01/wip-ifed-affiliation

.githubmap, .mailmap, .organizationmap, .peoplemap: update Igor

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

common/blkdev: fix build in FreeBSD environment

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>

.githubmap, .mailmap, .organizationmap, .peoplemap: update Igor
Fedotov's affiliation

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

doc: update docs to remove references of ubuntu 14.04

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/docker-test: drop support for ubuntu 14.04 and centos 6

* we cannot build on centos 6. and we are dropping the support for ubuntu
  14.04 also.
* add ubuntu 17.04: this release is not a LTS release, and added merely for
  demostration purpose.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/hadoop: switch from trusty to ubuntu latest

use the ubuntu_face.yaml facet for the distro used for testing. since we
are dropping the support for trusty.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/fs: use ubuntu_latest for multiclient tasks

instead of using ubuntu 14.04, since we want to drop the support of this
release.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/fs: use ubuntu_latest for libcephfs_java

* instead of using ubuntu 14.04, use ubuntu_latest.since we want
  to drop the support of this release.
* refactor this test to use the facet of ubuntu_latest.yaml.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks: remove test_buildpackages

the buildpackages feature is not part of ceph. and its test was moved to
teuthology in 5a43f8d5. let's remove the leftovers from ceph/ceph. BTW,
we have not run this test suite since then. and its dependency cannot be
fulfilled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: don't create snap inode during trans-authority rename

MDCache::cow_inode() may add auth pin to frozen inode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: trim non-auth snap dentry during cache rejoin

Because corresponding dentry in auth mds may have been pruned (snap deleted)

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: detach unlinked inode's snaprealm from its parent

When replaying slave update, inode may gets unlinked, but preserved
in memory in case of rollback. In this case, we need to detach the
inode's snaprealm from its parent.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

mds: fix CInode::find_snaprealm()

If inode has no parent dentry (newly created), CInode::find_snaprealm()
should try the oldest projected parent.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #19196 from shun-s/wip-cephdisk-fix-major-error

ceph-disk: fix signed integer is greater than maximum when call major

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19296 from shinobu-x/strange_double_space

osd: Remove double space

osd: Remove double space

Should not break anything but just strange...

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #19161 from xiexingguo/wip-pg-log

osd/PGLog: get rid of ineffective container operations

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PGLog: get rid of ineffective container operations

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #19242 from liewegas/wip-21147

qa/suites/rados/thrash: extend mgr beacon grace when many msgr failures injected

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19185 from liewegas/wip-trim-mgrmap

mon/MgrMonitor: limit mgrmap history

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19285 from ceph/wip-rm22299

ceph-volume Format correctly when vg/lv cannot be used

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #19259 from ceph/wip-rm22280

ceph-volume: handle leading whitespace/tabs in ceph.conf

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests verify proper formatting of RuntimeError on vg/lv error

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare correctly format argument for vg/lv error

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #19264 from theanalyst/doc/releases/12.2.2

doc: add 12.2.2 release notes

Reviewed-By: Sage Weil <sage@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>