Merge pull request #21128 from ricardoasmarques/wip-usage-bar

mgr/dashboard: Add usage bar component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21240 from rjfd/wip-dashboard-task-fixes

mgr/dashboard: TaskManager bug fixes

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21127 from smithfarm/wip-make-dist-py3

build/ops: make-dist: fall back to python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #21238 into master

* refs/pull/21238/head:
doc: fix typos and other minor errors in cephfs-nfs doc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21204 into master

* refs/pull/21204/head:
mds: PendingReleaseNotes and dev doc for open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20992 from cbodley/wip-dmclock-remove-by-ptr

dmclock: update mClockPriorityQueue with changes in subtree

Merge pull request #21261 from batrick/ceph-update-ceph-object-corpus

ceph-object-corpus: update to fix make check

ceph-object-corpus: update to fix make check

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21148 from wjwithagen/wjw-setup-virtualenv

tools: FreeBSD basic getopt does not use --options

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21237 from guzhongyan/zabbix-fix

mgr/zabbix: Fix wrong log message

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21253 from ceph/wip-yuriw-crontab

tests/qa - enabled `ceph-deploy` runs on `mira` nodes

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

Merge PR #21254 into master

* refs/pull/21254/head:
mds: bump mds_log_max_segments for trim buffer

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21252 from neha-ojha/wip-modify-crontab

qa/crontab: run the perf-basic suite every day

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

mds: bump mds_log_max_segments for trim buffer

Under create heavy workloads, the MDS will sometimes get behind trimming but
catch up. This avoids unnecessary warnings.

Fixes: http://tracker.ceph.com/issues/23560
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

tests/qa - enabled `ceph-deploy` runs on `mira` nodes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: Apply usage-bar component on health

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Apply usage-bar component on osd

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Apply usage-bar component on cephfs

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add usage bar component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

qa/crontab: run the perf-basic suite every day

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #21101 from dzafman/wip-23267

scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

build/ops: make-dist: fall back to python3

If python2.7 is not available, we might be building in a Python 3-only
environment.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21146 from zmc/wip-cephmetrics

qa/suites/cephmetrics: Updates for new version

Merge pull request #21244 from ifed01/wip-ifed-min-blob-descr

os/bluestore: update description for bluestore_compression_[min|max|_blob_size options

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: update description for
bluestore_compression_[min|max|_blob_size options

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mgr/dashboard: TaskManager: implement finished tasks as a list

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: fix exception serialization for tasks

Signed-off-by: Ricardo Dias <rdias@suse.com>

doc: fix typos and other minor errors in cephfs-nfs doc

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/zabbix: Fix wrong log message

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #21087 from ricardoasmarques/wip-date-pipe

mgr/dashboard: Add CdDatePipe

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21060 from ricardoasmarques/wip-view-cache-status-for

mgr/dashboard: Support aditional info on 'cd-view-cache'

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21045 from votdev/remove_useless_code

mgr/dashboard: Remove unused code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge PR #16779 into master

* refs/pull/16779/head:
mds: cleanup MDCache::open_snaprealms()
mds: make sure snaptable version > 0
mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
mds: replace MAX() with std::max()
tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
mds: don't warn unconnected snaplrealms in cluster log
mds: update CInode/CDentry's first according to global snapshot seq
qa/cephfs: add tests for snapclient cache
qa/cephfs: add tests for snaptable transaction
mds: add asok command that dumps cached snap infos
qa/cephfs: add tests for multimds snapshot
client: don't mark snap directory complete when its dirstat is empty
qa/workunits/snaps: add snaprealm split test
mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
client: fix incorrect snaprealm when adding caps
qa/workunits/snaps: add hardlink snapshot test
mds: add incompat feature and bump protocol for snapshot changes
mds: detach inode with single hardlink from global snaprealm
mds: record hardlink snaps in inode's snaprealm
mds: attach inode with multiple hardlinks to dummy global snaprealm
mds: cleanup rename code
mds: ensure xlocker has uptodate lock state
mds: simplify SnapRealm::build_snap_{set,trace}
mds: record global last_created/last_destroyed in snaptable
mds: pop projected snaprealm before inode's parent changes
mds: keep isnap lock in sync state
mds: handle mksnap vs resolve_snapname race
mds: cleanup snaprealm past parents open check
mds: rollback snaprealms when rolling back slave request
mds: send updated snaprealms along with slave requests
mds: explict notification for snap update
mds: send snap related messages centrally during mds recovery
mds: synchronize snaptable caches when mds recovers
mds: introduce MDCache::maybe_finish_slave_resolve()
mds: notify all mds about prepared snaptable update
mds: record snaps in old snaprealm when moving inode into new snaprealm
mds: cache snaptable in snapclient
mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/cephmetrics: Add new facets

For EPEL and containerized prometheus/grafana

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/cephmetrics: Update ceph-ansible config

This is taken from the ceph-ansible suite.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge PR #20915 into master

* refs/pull/20915/head:
Add config doc for exporting CephFS over NFS

Reviewed-by: Jeff Layton <jlayton@redhat.com>

osd: Fix stale scrub stats when a primary takes over

Fixes: http://tracker.ceph.com/issues/23267
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #21003 into master

* refs/pull/21003/head:
client: lookup . on non-directory inode
client: avoid may_lookup for lookup . and lookup ..

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #21221 into master

* refs/pull/21221/head:
client: fix _read return for invalid offset

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

client: fix _read return for invalid offset

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21088 into master

* refs/pull/21088/head:
client: fixup warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21050 from falcon78921/wip-doc-falcon78921

doc: updated add primary storage documentation for latest CloudStack release (4.11)

reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #21218 from ceph/wip-dgall-crontab

test:qa:infra - Run update daily and use bash

test:qa:infra - Run update daily and use bash

bash has the path for `source` to activate the virtualenv.  sh doesn't.

[skip ci]

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge tag 'v13.0.2'

v13.0.2

mgr/dashboard: Add CdDatePipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #21180 into master

* refs/pull/21180/head:
vstart_runner: examine check_status before error

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20460 from colletj/v1_image_creation_disallow

librbd: disallow creation of v1 image format

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21202 from tchaikov/wip-rbd-replay

rbd-replay: remove boost dependency

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21142 from dragonylffly/wip-fix-ebusy

rbd-nbd: fix ebusy when do map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21157 from trociny/wip-23526

journal: limit number of appends sent in one librados op

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds: PendingReleaseNotes and dev doc for open file table

Fixes: http://tracker.ceph.com/issues/23532
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

rbd-replay: remove boost dependency

quite a few facilies are now available in standard library now after we
switched to C++17.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Remove useless code

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge PR #20855 into master

* refs/pull/20855/head:
client: add the fuse parameter max_write

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21096 into master

* refs/pull/21096/head:
pybind/cephfs: added comments to cephfs.pyx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21183 from neha-ojha/wip-minor-fix-perf-suite

qa/suites/perf-basic: add desc regarding test machines

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21075 from ceph/wip-yuriw-try-crontab

test:qa:infra - teuthology crontab items as of 3/27/18

qa/suites/perf-basic: add desc regarding test machines

Signed-off-by: Neha Ojha <nojha@redhat.com>

journal: limit number of appends sent in one librados op

Fixes: http://tracker.ceph.com/issues/23526
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21165 from Songweibin/wip-drop-redefine-func

librbd: minor code cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

vstart_runner: examine check_status before error

Issue found by Rishabh Dave. Fix suggested by John Spray.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

CMakeLists: v13.0.2

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21143 from wjwithagen/wjw-ipaddr-ipv6

common: FreeBSD wants the correct struct selection for ipv6

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21136 from kungf/io_events

bluestore: fix aio_t::rval  type

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20820 from guzhongyan/nodels-improve

mon: node ls improvement

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20298 from markhpc/wip-osd-check_full

osd: don't process ostream strings when not debugging.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Add config doc for exporting CephFS over NFS

... using NFS-Ganesha NFS server.

Fixes: https://tracker.ceph.com/issues/23271
Signed-off-by: Ramana raja <rraja@redhat.com>

Merge pull request #21159 from tchaikov/wip-no-more-use-wheel

src: s/pip --use-wheel/pip/

Reviewed-by: Nathan Cutler <ncutler@suse.com>

pybind/rbd: normalize errno exception

The code meant to do so but actually it does not ...

Signed-off-by: songweibin <song.weibin@zte.com.cn>

librbd: return error when r < 0

Signed-off-by: songweibin <song.weibin@zte.com.cn>

librbd: drop a duplicate definition function

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src: s/--use-wheel//

--use-wheel was deprecated in favor of --only-binary in pip v7.0.0. and
--use-wheel was removed in a recent release of pip. but some packages
are source packages, so we cannot simply replace use-wheel with
only-binary. so a simpler approach is to drop --use-wheel option, as pip
respects --find-links, and will find the required package from the
wheelhouse.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools: FreeBSD basic getopt does not use --options

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #20132 into master

* refs/pull/20132/head:
qa/cephfs: update TestDamage for open file table
mds: allow storing open file table in multiple omaps
mds: differentiate Anchor types to clarify purpose
mds: add perf counter for 'open ino' operation
mds: protect open file table against partial omap update
mds: add dirfrags whose child inodes have caps to open file table
mds: don't try prefetching destroyed inodes
mds: don't try opening inodes that haven't been created
mds: don't re-requeue open files to head of log
mds: use open file table to speed up mds recovery
mds: introduce open file table
mds: track how many clients/mds want caps for each inode
mds: cleanup MDCache::opening_inodes access
mds: cleanup CInode/CDir states definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21147 from rjfd/wip-dashboard-fix-notification-queue-test

mgr/dashboard: notification queue: fix priority tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: notification queue: fix priority tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20979 from ceph/wip-cd-smoke-fix

qa/tests: use ceph-deploy stable branch for single node tests

qa/tests: use stable branch for single node tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

common: FreeBSD wants the correct struct selection for ipv6

Lets see if this also works for Linux

Fixes: http://tracker.ceph.com/issues/21813
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

rbd-nbd: fix ebusy when do map

When doing rbd-nbd map, if the Ceph service is not available,
the codes will wait on rados.connect(), unless killing the process.
In that case, the close_nbd logic is skipped with NBD_CLEAR_SOCK ioctl
not called. On the CentOS 7 kernel, it leaves nbd->file not cleared, which
causes the subsequent map requests return EBUSY, this patch fixes it
by connecting Ceph first prior to calling NBD_SET_SOCK ioctl

Fixes: http://tracker.ceph.com/issues/23528
Signed-off-by: Li Wang <laurence.liwang@gmail.com>

librbd/cli-integration: Permit V1 image creation

Signed-off-by: Julien COLLET <julien.collet@cern.ch>

Merge pull request #18317 from hitoshikamei/master-rbd

rbd: eager-thick provisioning support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20868 from shun-s/fix-same-cookie

librbd: fix refuse to release lock when cookie is the same at rewatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20935 from trociny/wip-get-name

librbd: add API function to get image name

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: change the type of aio_t:res to long

io_event::res was assigned to io_t::rval, but io_event::res
is long, if rval is int, this will cause overflow.

Fixes: http://tracker.ceph.com/issues/23527
Signed-off-by: kungf <yang.wang@easystack.cn>

Merge pull request #21117 from smithfarm/wip-23504

tests: remove TestPGLog ASSERT_DEATH test

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #20927 into master

* refs/pull/20927/head:
doc: use actual entity address for clarity
doc: make minor grammatical rectifications

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21131 from trociny/wip-23502

qa/suites/rbd: set qemu task time_wait param

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: Edits

Made a few minor edits.

Signed-off-by: John Wilkins <jowilkin@redhat.com>

doc: Updated add primary storage documentation for latest CloudStack release (4.11)

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

osd/OSD: make dout in _check_full conditional

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge PR #20983 into master

* refs/pull/20983/head:
ceph_authtool: add mode option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph_authtool: add mode option

We now have the ability to set the keyring file mode with the help of
'--mode MODE'. The mode needs to be specified in octal using the format:
0600.

Closes: http://tracker.ceph.com/issues/23513
Signed-off-by: Sébastien Han <seb@redhat.com>

qa/suites/rbd: set qemu task time_wait param

so workloads qemu_dynamic_features.sh and qemu_rebuild_object_map.sh,
which check if qemu is finished with periodicity 60 sec, have enough
time to detect this before the rbd image is removed.

Fixes: https://tracker.ceph.com/issues/23502
Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished

In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.

Signed-off-by: Mykola Golub <mgolub@suse.com>

test:qa:infra - teuthology crontab items as of 3/27/18

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21121 from tchaikov/wip-no-relicensing

msg/async/dpdk: remove xsky copyright and LGPL copying

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #21105 from ashitakasam/wip-bug

rgw: comparison between signed and unsigned integer expressions

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21074 from trociny/wip-librados_test_stub-append

test/librados_test_stub: always create copy of buffers passed to operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>