ceph.in: drop semicolons; relax priority arg ordering

Priority had to be the last of the non-numeric arguments,
or it'd be overwritten

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/ceph_daemon.py: _gettermsize fails on non-terminals

default to 80x25; this should be for terminalless tests only

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.spec.in, debian/control: add dep for python-prettytable

mgr/pybind/fsstatus already required this, but ceph.in does now.
Add as a build dependency as well to handle tests.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in: daemonperf: add 'ls' as a synonym for 'list'

Signed-off-by: Dan Mick <dan.mick@redhat.com>

osd: clean up perf counter priorities whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: set priorities

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: set priorities

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/ceph_daemon: preserve ordering

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/ceph_daemon: default to prio 0 if unspecified

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.in, pybind/ceph_daemon.py: add 'daemonperf list'

respect statpats and priority; display as a PrettyTable

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in, pybind/ceph_daemon.py: add priority filtering

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/ceph_daemon.py: allow sections to split for 'fitting'

Relax requirement that entire section fit; show whatever stats
fit out of the sections selected.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in, pybind/ceph_daemon.py: allow stat filter patterns

optional second argument to ceph

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/ceph_daemon.py: reprint headers as soon as they disappear

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/ceph_daemon.py: stop early termination of sleep

Any unignored signal will EINTR underneath and cause early
return of sleep().  Fix that by sleeping until end time is
reached.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/ceph_daemon.py: DaemonWatcher: fit display to window size

Keep track of terminal width/height, and output as many
candidate statistics as will fit in the window.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in: Move daemonperf to its own function

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in: move daemon commands to their own function

Signed-off-by: Dan Mick <dan.mick@redhat.com>

common/perf_counters: replace suppress_nicks with prio adjustment

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: set perf counter priorities

Signed-off-by: Sage Weil <sage@redhat.com>

mds: add perf counter nicks, priorities

Signed-off-by: Sage Weil <sage@redhat.com>

osd: set perf counter priorities

Signed-off-by: Sage Weil <sage@redhat.com>

osd: clean up whitespace in create_logger()

Signed-off-by: Sage Weil <sage@redhat.com>

common/perf_counters: allow priorities to be assigned to perf counters

The idea is that higher priority counters can be surfaced by user tools
automatically (e.g., 'ceph daemonperf') without enshrining knowledge in
the tool of the internal workings of the daemon.  Similarly, the tool
may not know which counters are in use for a particular daemon (e.g.,
filestore vs bluestore).

This requires a bit of discipline in the ceph code to use consistent
priorities, but as long as developers are using the tools that consume
the priorities (and I expect they will if we make the counters useful)
we should converge on good values.

Signed-off-by: Sage Weil <sage@redhat.com>

common/perf_counters: require nicks be <= 4 chars

This keeps the column sizes at 5 chars for 'ceph daemonperf'.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: rename nicks to be <= 4 chars

Signed-off-by: Sage Weil <sage@redhat.com>

osd: rename nicks to be <= 4 chars

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: rename nicks to be <= 4 chars

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14783 from dachary/wip-crush-classe-test

test: use 7130 for crush-classes.sh

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14651 from tchaikov/wip-silent-boost-build

cmake: pass -d0 to b2 if not CMAKE_VERBOSE_MAKEFILE

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14731 from ceph/wip-bluestore-type

os/bluestore: use correct bound encode size for unused

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14738 from liupan1111/wip-fix-initial

src/common: change last_work_queue to next_work_queue.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14757 from dmick/wip-einval

common: Better handling for missing/inaccessible ceph.conf files

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14764 from Liuchang0812/wip-strip-pglog-op-name

osd: strip pglog op name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14772 from majianpeng/bluestore-remove-extentfreelist

os/bluestore: Remove ExtentFreeListManager.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14777 from tchaikov/wip-19398

osd/PGMonitor: always update pgmap with latest osdmap

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: Remove ExtentFreeListManager.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test: use 7130 for crush-classes.sh

7127 is already used by mon/misc.sh

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #14350 from batrick/maintainer-update

AUTHORS: update tech leads

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14352 from batrick/peoplemap-update

peoplemap: add pdonnell alias

Merge pull request #14494 from drunkard/master

doc: two minor fixes

Merge pull request #14493 from johnzzpcrystal/FileJournal-typo

os/filestore/FileJournal: Fix typo in the comment

Merge pull request #14503 from wjwithagen/wip-wjw-freebsd-doc-dev

doc/dev: add some info about FreeBSD

Merge pull request #14560 from wjwithagen/wip-wjw-freebsd-init-ceph

src/init-ceph.in:  allow one((re)?start|stop) as commands

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14625 from theanalyst/doc/releases/12.0.2

doc: v12.0.2 (dev) release notes

Merge pull request #14713 from smithfarm/wip-19740

tests: set -x in suites/iozone.sh workunit

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14756 from yehudasa/wip-rgw-suite-fix

qa/suite: replace reference to fs/xfs.yaml

Merge pull request #14747 from Liuchang0812/wip-mailmap

mailmap: update organization info

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #14718 from liewegas/wip-bluestore-tool

ceph-kvstore-tool: allow 'bluestore-kv' as kvdb type; add escaping, compaction

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

osd/PGMonitor: always update pgmap with latest osdmap

without this change, we always update the pgmap with the latest osdmap in
PGMonitor::check_osd_map(), but assign pending_inc.osdmap_epoch with the
epoch passed to PGMonitor::check_osd_map(), even if the osdmap could be
be much newer than "epoch". for example, we could end up with an
MOSDPGCreate with an ancient epoch, but carrying some creating pgs whose
pools do not exist in osdmap of that ancient epoch yet. this happens
when RetryCheckOSDMap is involved.

Fixes: http://tracker.ceph.com/issues/19398
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14725 from tchaikov/wip-cmake

cmake: do not compile crush twice

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14730 from tchaikov/wip-19744

mon/OSDMonitor: update pg_creatings even the new acting set is empty

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14744 from ceph/wip-bluefs-alloc-size

os/bluestore: align reclaim size to bluefs_alloc_size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14749 from ifed01/wip-bluestore-nolock-alloc

os/bluestore: get rid off excessive lock at BitMapAllocator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14762 from zy751713126/add-bench-notes

rbd/bench: add notes of default values, it's easy to use

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd/bench: add notes of default values, it's more easy to use

Signed-off-by: Zheng Yin <zhengyin@chinac.com>

src/common: change last_work_queue to next_work_queue.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

mailmap: update organization info

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

squashme: review comments

Signed-off-by: Dan Mick <dan.mick@redhat.com>

test/cli/ceph-conf/env-vs-args.t: accept new error messages

Failing to load ceph.conf now dumps more errors for the user

Signed-off-by: Dan Mick <dan.mick@redhat.com>

test/librados_test_stub/LibradosTestStub.cc: accept ENOENT

Missing ceph.conf now returns ENOENT rather than EINVAL

Signed-off-by: Dan Mick <dan.mick@redhat.com>

global/global_init.cc global_pre_init: look for ENOENT

EINVAL for no conf file found changes to ENOENT

Signed-off-by: Dan Mick <dan.mick@redhat.com>

osd/pglog: align pglog op name in ostream

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

osd/pglog: strip pglog op name

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

librados/librados.cc rados_conf_read_file: report errors to client

Previously all the error text was hidden; log it.
Note: ENOENT is noted as a 'parse error'

Signed-off-by: Dan Mick <dan.mick@redhat.com>

common/config.cc md_config_t::parse_config_files: return ENOENT

If the only error we get is ENOENT, pass that back instead of
EINVAL, as it more-accurately reflects the error(s)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

common/ConfUtils.cc parse_file: fix function name in errors

Signed-off-by: Dan Mick <dan.mick@redhat.com>

common/ConfUtils.cc: ConfFile:parse_file: warn about ENOENT

Consider ENOENT an error; it may or may not be reported, but
accumulate it as if it might be

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.in, pybind/rados/rados.pyx:  Handle EINVAL better

Translate EINVAL to an exception type in rados.pyx
Print entire exception in ceph CLI

Signed-off-by: Dan Mick <dan.mick@redhat.com>

qa/suite: replace reference to fs/xfs.yaml

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14726 from batrick/i19583

mds: change_attr++ and set ctime for set_vxattr

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14652 from jcsp/wip-18816

mds: remove "mds log" config option

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14636 from jcsp/wip-19691

osdc: remove journaler_allow_split_entries option

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14317 from ukernel/wip-19306

client: make seeky readdir more efficiency

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12935 from stiopaa1/17855_evictClient

mds/Server.cc: Don't evict a slow client if...

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14388 from shashalu/fix-create-bucket

rgw: using the same bucket num_shards as master zg when create bucket in secondary zg

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: v12.0.2 (dev) release notes

Also adding the commits after 12.0.1 tag merged back into the master

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #14479 from swamireddy/v12.0.1_credits

mailmap: V12.0.1 credits

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14611 from liewegas/wip-mon-warn-cleanup

mon/PGMonitor: clean up min/max span warning

Reviewed-by: David Zafman <dzafman@redhat.com>

ceph-kvstore-tool: compact{,-prefix,-range}

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-kvstore-tool: (url) escape prefixes and keys

Escape output; unescape input.

Signed-off-by: Sage Weil <sage@redhat.com>

common/url_escape: add simple url_[un]escape() methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: get rid off excessive lock at BitMapAllocator

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

ceph-kvstore-tool: tabs instead of : as delimiter

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-kvstore-tool: nicer failure message on open failure

I have been hating this assert for a long time.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-kvstore-tool: 'bluestore-kv' to open bluestore's embedded kv

This is usually rocksdb on bluefs, but it should work with whatever kv
backend bluestore is configured to use.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-bluestore-tool: add error handling to bluefs-export

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: BLUEFS_START -> SUPER_RESERVED

We reserve the first 8k for bluestore and bluefs superblocks even if
bluefs isn't enabled at mkfs time.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14741 from trociny/wip-19405

test/rbd_mirror: race in TestMockLeaderWatcher.AcquireError

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12792 from david-z/wip-mds-use-tcmalloc

mds: remove boost::pool usage and use tcmalloc directly

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14575 from jcsp/wip-cephfs-suites

qa: Tidy up fs/ suite

Reviewed-by: Yan, Zheng <zyan@redhat.com>

qa: switch off btrfs for multimds and kcephfs

Signed-off-by: John Spray <john.spray@redhat.com>

qa: update links for modified fs subsuite names

Signed-off-by: John Spray <john.spray@redhat.com>

qa/tasks/cephfs: clean up mount point setup

Previously were sometimes trying to maintain a mounted
client across a filesystem destroy/create.

Signed-off-by: John Spray <john.spray@redhat.com>

qa/suites: don't use btrfs for cephfs testing

This change happened a while back, but it got rolled back
when the generic objectstore/ dir had its filestore
entry split out into xfs and btrfs in 208675af.

Signed-off-by: John Spray <john.spray@redhat.com>

qa: remove unused fs/ dir

The objectstore/ dir is now selecting the backing fs.

Signed-off-by: John Spray <john.spray@redhat.com>

qa/suites/fs: more meaningful names for sub suites

The "recovery" sub suite was originally tests for
client/mds recovery in certain failure cases, it has
since grown to include lots of unit testing of
various features using CephFSTestCase.

The "basic" suite is now specifically just running workloads
now that I've moved out the smaller functional tests.

Signed-off-by: John Spray <john.spray@redhat.com>

qa/suites/fs: put simple functional tests together

Most of what's in basic/ is "workload" type testing
(i.e. a simple cluster cluster configuration and then
running a script inside the client), which gets
permuted in various ways.  Move the simpler
functional tests out with the others like themselves.

Signed-off-by: John Spray <john.spray@redhat.com>

qa: don't permute object store for functional tests

These are unit tests for specific CephFS functionality,
it is gratuitous to repeat them with different underlying
RADOS object stores.

We retain coverage of XFS vs. bluestore in the workload tests.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #14018 from jcsp/wip-17939

client: getattr before returning quota/layout xattrs

Reviewed-by: Yan, Zheng <zyan@redhat.com>