osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max

The _max isn't used.  Drop the _min suffix.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21250 from liewegas/wip-recovery-slow-ops

qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS

qa/suites/rados/singleton/all/ec-lost-unfound: whitelist SLOW_OPS

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21239 from rjfd/wip-dashboard-routes

mgr/dashboard: using RoutesDispatcher as HTTP request dispatcher

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #19331 from jecluis/wip-mon-osdmap-prune

mon: osdmap prune

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20464 from cbodley/wip-rgw-beast-ssl

rgw: add ssl support to beast frontend

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #21128 from ricardoasmarques/wip-usage-bar

mgr/dashboard: Add usage bar component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21240 from rjfd/wip-dashboard-task-fixes

mgr/dashboard: TaskManager bug fixes

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

qa: enable mon osdmap pruning on 'rados/' suites

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa: enable mon osdmap pruning on rados/monthrash

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa: mon: osdmap pruning standalone/workunit

Keep a standalone wrapper for the workunit, so we can test it locally,
leveraging the ceph-helpers to do the setup. Keep a workunit to be
exercised by teuthology.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/standalone: change PATH to allow finding sysctl

If, by default, a user's PATH does not contain /sbin and /usr/sbin, we
may have a hard time finding sysctl to adjust the kernel core pattern.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/standalone: deduplicate cleanup code

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/standalone: discard pipe in sysctl kernel.core_pattern

At least verified in openSUSE tumbleweed and ubuntu 16.04,
`sysctl -n kernel.core_pattern` returns a pipe at the start of the
value. We reset core_pattern to its original form once the script is
about to end, but if we do not discard the pipe the new value will
contain an extra pipe (apparently, the pipe is added automatically at
some point, possibly simply on read). We are simply stripping it on
read, as to prevent this behavior.

Additionally, we are also enclosing the reset of kernel.core_pattern in
quotes, so as to make sure patterns that include spaces (e.g., ubuntu's
apport, or tumbleweed's systemd-coredump) are properly reset.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: allow triggering prunes from tick()

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: allow blocking trim if option is enabled

We need to allow blocking osdmap trimming to test osdmap pruning.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: dump osdmap manifest alongside other infos

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon/OSDMonitor: prune full maps on encode_pending

If we have gone over a safety threshold for number of versions, start
pruning full osdmaps, poking holes in the sequence. To make up for the
missing maps, we will rebuild full maps based on the incremental maps
and the closest pinned full map we have available.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

doc: add dev notes for mon's full osdmap pruning

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21127 from smithfarm/wip-make-dist-py3

build/ops: make-dist: fall back to python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #21238 into master

* refs/pull/21238/head:
doc: fix typos and other minor errors in cephfs-nfs doc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21204 into master

* refs/pull/21204/head:
mds: PendingReleaseNotes and dev doc for open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: beast frontend calls shutdown before closing connections

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: verify suite tests beast with ssl

instead of adding special frontend configs like civetweb_ssl.yaml and
beast_ssl.yaml, added a new proto/ subdirectory for http.yaml and
https.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add ssl support to beast frontend

adds frontend options ssl_certificate, ssl_private_key, ssl_port, ssl_endpoint

Fixes: http://tracker.ceph.com/issues/22832
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: factor the tcp::socket out of ClientIO

remove ClientIO's dependency on a concrete socket type by moving it into
a derived StreamIO class in rgw_asio_frontend.cc

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20992 from cbodley/wip-dmclock-remove-by-ptr

dmclock: update mClockPriorityQueue with changes in subtree

Merge pull request #21261 from batrick/ceph-update-ceph-object-corpus

ceph-object-corpus: update to fix make check

ceph-object-corpus: update to fix make check

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21148 from wjwithagen/wjw-setup-virtualenv

tools: FreeBSD basic getopt does not use --options

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21237 from guzhongyan/zabbix-fix

mgr/zabbix: Fix wrong log message

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21253 from ceph/wip-yuriw-crontab

tests/qa - enabled `ceph-deploy` runs on `mira` nodes

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

Merge PR #21254 into master

* refs/pull/21254/head:
mds: bump mds_log_max_segments for trim buffer

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21252 from neha-ojha/wip-modify-crontab

qa/crontab: run the perf-basic suite every day

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

mds: bump mds_log_max_segments for trim buffer

Under create heavy workloads, the MDS will sometimes get behind trimming but
catch up. This avoids unnecessary warnings.

Fixes: http://tracker.ceph.com/issues/23560
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

tests/qa - enabled `ceph-deploy` runs on `mira` nodes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: Apply usage-bar component on health

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Apply usage-bar component on osd

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Apply usage-bar component on cephfs

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add usage bar component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

qa/crontab: run the perf-basic suite every day

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS

Recovery and peering can be slow enough with all the logging enabled to
trigger a slow ops warning.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21101 from dzafman/wip-23267

scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

build/ops: make-dist: fall back to python3

If python2.7 is not available, we might be building in a Python 3-only
environment.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: move all controller infrastructure to controllers/__init__.py

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: using RoutesDispatcher as HTTP request dispatcher

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21146 from zmc/wip-cephmetrics

qa/suites/cephmetrics: Updates for new version

Merge pull request #21244 from ifed01/wip-ifed-min-blob-descr

os/bluestore: update description for bluestore_compression_[min|max|_blob_size options

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: update description for
bluestore_compression_[min|max|_blob_size options

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mgr/dashboard: TaskManager: implement finished tasks as a list

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: fix exception serialization for tasks

Signed-off-by: Ricardo Dias <rdias@suse.com>

doc: fix typos and other minor errors in cephfs-nfs doc

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/zabbix: Fix wrong log message

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #21087 from ricardoasmarques/wip-date-pipe

mgr/dashboard: Add CdDatePipe

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21060 from ricardoasmarques/wip-view-cache-status-for

mgr/dashboard: Support aditional info on 'cd-view-cache'

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21045 from votdev/remove_useless_code

mgr/dashboard: Remove unused code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge PR #16779 into master

* refs/pull/16779/head:
mds: cleanup MDCache::open_snaprealms()
mds: make sure snaptable version > 0
mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
mds: replace MAX() with std::max()
tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
mds: don't warn unconnected snaplrealms in cluster log
mds: update CInode/CDentry's first according to global snapshot seq
qa/cephfs: add tests for snapclient cache
qa/cephfs: add tests for snaptable transaction
mds: add asok command that dumps cached snap infos
qa/cephfs: add tests for multimds snapshot
client: don't mark snap directory complete when its dirstat is empty
qa/workunits/snaps: add snaprealm split test
mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
client: fix incorrect snaprealm when adding caps
qa/workunits/snaps: add hardlink snapshot test
mds: add incompat feature and bump protocol for snapshot changes
mds: detach inode with single hardlink from global snaprealm
mds: record hardlink snaps in inode's snaprealm
mds: attach inode with multiple hardlinks to dummy global snaprealm
mds: cleanup rename code
mds: ensure xlocker has uptodate lock state
mds: simplify SnapRealm::build_snap_{set,trace}
mds: record global last_created/last_destroyed in snaptable
mds: pop projected snaprealm before inode's parent changes
mds: keep isnap lock in sync state
mds: handle mksnap vs resolve_snapname race
mds: cleanup snaprealm past parents open check
mds: rollback snaprealms when rolling back slave request
mds: send updated snaprealms along with slave requests
mds: explict notification for snap update
mds: send snap related messages centrally during mds recovery
mds: synchronize snaptable caches when mds recovers
mds: introduce MDCache::maybe_finish_slave_resolve()
mds: notify all mds about prepared snaptable update
mds: record snaps in old snaprealm when moving inode into new snaprealm
mds: cache snaptable in snapclient
mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/cephmetrics: Add new facets

For EPEL and containerized prometheus/grafana

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/cephmetrics: Update ceph-ansible config

This is taken from the ceph-ansible suite.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge PR #20915 into master

* refs/pull/20915/head:
Add config doc for exporting CephFS over NFS

Reviewed-by: Jeff Layton <jlayton@redhat.com>

osd: Fix stale scrub stats when a primary takes over

Fixes: http://tracker.ceph.com/issues/23267
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #21003 into master

* refs/pull/21003/head:
client: lookup . on non-directory inode
client: avoid may_lookup for lookup . and lookup ..

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #21221 into master

* refs/pull/21221/head:
client: fix _read return for invalid offset

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

client: fix _read return for invalid offset

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21088 into master

* refs/pull/21088/head:
client: fixup warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21050 from falcon78921/wip-doc-falcon78921

doc: updated add primary storage documentation for latest CloudStack release (4.11)

reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #21218 from ceph/wip-dgall-crontab

test:qa:infra - Run update daily and use bash

test:qa:infra - Run update daily and use bash

bash has the path for `source` to activate the virtualenv.  sh doesn't.

[skip ci]

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge tag 'v13.0.2'

v13.0.2

mgr/dashboard: Add CdDatePipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #21180 into master

* refs/pull/21180/head:
vstart_runner: examine check_status before error

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20460 from colletj/v1_image_creation_disallow

librbd: disallow creation of v1 image format

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21202 from tchaikov/wip-rbd-replay

rbd-replay: remove boost dependency

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21142 from dragonylffly/wip-fix-ebusy

rbd-nbd: fix ebusy when do map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21157 from trociny/wip-23526

journal: limit number of appends sent in one librados op

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds: PendingReleaseNotes and dev doc for open file table

Fixes: http://tracker.ceph.com/issues/23532
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

rbd-replay: remove boost dependency

quite a few facilies are now available in standard library now after we
switched to C++17.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Remove useless code

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge PR #20855 into master

* refs/pull/20855/head:
client: add the fuse parameter max_write

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21096 into master

* refs/pull/21096/head:
pybind/cephfs: added comments to cephfs.pyx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21183 from neha-ojha/wip-minor-fix-perf-suite

qa/suites/perf-basic: add desc regarding test machines

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21075 from ceph/wip-yuriw-try-crontab

test:qa:infra - teuthology crontab items as of 3/27/18

qa/suites/perf-basic: add desc regarding test machines

Signed-off-by: Neha Ojha <nojha@redhat.com>

journal: limit number of appends sent in one librados op

Fixes: http://tracker.ceph.com/issues/23526
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21165 from Songweibin/wip-drop-redefine-func

librbd: minor code cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

vstart_runner: examine check_status before error

Issue found by Rishabh Dave. Fix suggested by John Spray.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

CMakeLists: v13.0.2

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21143 from wjwithagen/wjw-ipaddr-ipv6

common: FreeBSD wants the correct struct selection for ipv6

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21136 from kungf/io_events

bluestore: fix aio_t::rval  type

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20820 from guzhongyan/nodels-improve

mon: node ls improvement

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20298 from markhpc/wip-osd-check_full

osd: don't process ostream strings when not debugging.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Add config doc for exporting CephFS over NFS

... using NFS-Ganesha NFS server.

Fixes: https://tracker.ceph.com/issues/23271
Signed-off-by: Ramana raja <rraja@redhat.com>

Merge pull request #21159 from tchaikov/wip-no-more-use-wheel

src: s/pip --use-wheel/pip/

Reviewed-by: Nathan Cutler <ncutler@suse.com>

pybind/rbd: normalize errno exception

The code meant to do so but actually it does not ...

Signed-off-by: songweibin <song.weibin@zte.com.cn>

librbd: return error when r < 0

Signed-off-by: songweibin <song.weibin@zte.com.cn>

librbd: drop a duplicate definition function

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src: s/--use-wheel//

--use-wheel was deprecated in favor of --only-binary in pip v7.0.0. and
--use-wheel was removed in a recent release of pip. but some packages
are source packages, so we cannot simply replace use-wheel with
only-binary. so a simpler approach is to drop --use-wheel option, as pip
respects --find-links, and will find the required package from the
wheelhouse.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools: FreeBSD basic getopt does not use --options

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #20132 into master

* refs/pull/20132/head:
qa/cephfs: update TestDamage for open file table
mds: allow storing open file table in multiple omaps
mds: differentiate Anchor types to clarify purpose
mds: add perf counter for 'open ino' operation
mds: protect open file table against partial omap update
mds: add dirfrags whose child inodes have caps to open file table
mds: don't try prefetching destroyed inodes
mds: don't try opening inodes that haven't been created
mds: don't re-requeue open files to head of log
mds: use open file table to speed up mds recovery
mds: introduce open file table
mds: track how many clients/mds want caps for each inode
mds: cleanup MDCache::opening_inodes access
mds: cleanup CInode/CDir states definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21147 from rjfd/wip-dashboard-fix-notification-queue-test

mgr/dashboard: notification queue: fix priority tests

Reviewed-by: Kefu Chai <kchai@redhat.com>