mgr/dashboard: Fix growing table in firefox

It seems Firefox 52 has some kind of problem when
rendering our table inside a fieldset.

Since we don't really require the fieldset in the monitor page,
I have removed it to "fix" this problem.

Fixes: http://tracker.ceph.com/issues/26999
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23689 from tchaikov/wip-run-make-check-more-aio

run-make-check: increase fs.aio-max-nr to 1048576

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

Merge pull request #23675 from tserlin/tserlin-require-fuse

debian/control: require fuse for ceph-fuse

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23655 from tchaikov/wip-silence-Wsign-compare-warnings

test: silence -Wsign-compare warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

run-make-check: increase fs.aio-max-nr to 1048576

the solution was suggested by Yingxin Cheng. otherwise we could have
EAGAIN returned by io_setup(2).

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23630 from wjwithagen/wjw-fix-rbd-ggate-kldload

test/rbd: rbd_ggate test improvements

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23684 from tchaikov/wip-doc-balancer

doc/rados/operations: add balancer.rst to TOC

Reviewed-by: John Spray <john.spray@redhat.com>

doc/rados/operations: add balancer.rst to TOC

this helps user to find the balancer feature, and
also addresses the FTBFS of doc like

Warning, treated as error:
/home/jenkins-build/build/workspace/ceph-pr-docs/doc/rados/operations/balancer.rst:document
isn't included in any toctree

this doc was moved from doc/mgr/balancer.rst to
doc/rados/operations/balancer.rst in
a78d600e6b368255096d6f0bbee8e53553307118

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23642 from tchaikov/wip-crimson-msgr

cmake,crimson/net: add keepalive support, and enable unittest_seastar_messenger in "make check"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #23635 into master

* refs/pull/23635/head:
mds: use monotonic clock in beacon
mds: simplify beacon init

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21114 from colletj/colletj_rbd_image_timestamps

librbd: add rbd image access/modified timestamps

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: add image access/last modified timestamps

Add access and modify timestamps and associated tests
to RBD images.

Access (resp. modify) timestamps are updated on
read (resp. write) operations. A configurable throttling
mechanism is implemented (default to 60s).

Signed-off-by: Julien Collet <julien.collet@cern.ch>

debian/control: require fuse for ceph-fuse

This is the Ubuntu equivalent for:

https://github.com/ceph/ceph/pull/17120
http://tracker.ceph.com/issues/21057

Installing "ceph-fuse" should pull in the "fuse" package automatically.

Signed-off-by: Thomas Serlin <tserlin@redhat.com>

Merge pull request #23649 from trociny/wip-26939

librbd: journaling unable request can not be sent to remote lock owner

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23311 from Songweibin/wip-fix-du-display

tools/rbd: minor fixes for rbd du display

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23638 from Songweibin/wip-filter-user-snapshots

rbd: filter out group/trash snapshots from snap_list

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23615 from dmick/wip-ceph-mgr-deb

debian/rules: fix ceph-mgr .pyc files left behind

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23606 from ifed01/wip-ifed-stupid-get-fragmentation-fix

os/bluestore: fix assertion in StupidAllocator::get_fragmentation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23587 from xiexingguo/wip-recover-order

osd/PrimaryLogPG: optimize recover order

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23558 from liewegas/wip-mgr-always-on

mgr: keep status, balancer always on

Merge pull request #23573 from rzarzynski/wip-msg-26937

msg: entity_addr_t::parse doesn't do memset(this, 0, ...) for clean-up

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23641 from wjwithagen/wjw-fix-pthread_get_name_np

include/compat.h: make pthread_get_name_np work when available

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #23658 into master

* refs/pull/23658/head:
qa: use cephfs standard configs for kclient

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23657 from trociny/wip-wsign-compare

test/librbd: fix compiler -Wsign-compare warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23591 from tchaikov/wip-lockless-auth

auth,common: add lockless auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23567 from tchaikov/wip-cmake-cleanup

cmake: fix Debug build `WITH_SEASTAR=ON`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23631 from cyx1231st/test-seastar-config

crimson/common: apply config changes also on shard.0

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/test: improve test_config with observers

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge pull request #23582 from badone/wip-remove-upstart

Remove upstart files and references

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/rbd: Improve/update rbd-ggate

* check if geom_gate can be loaded before doing the actual tests

Otherwise continuing does not make sense.
Major reason for this problem is due to mismatch between
kernel and module versions.

* After FreeBSD kernevel 1200078 ggate resizing is possible

So set the flag that resizing can be tested

* Only sudo commands that really need sudo

rbd-ggate list is available in regular user mode

* be a bit more verbose during testing and list the test purpose

* list-mapped is an option in rbd-nbd, not (yet) in rbd-ggate

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

cmake: should use lowercase for component name in FindSanitizers.cmake

and s/undefined-behavior/undefined_behavior/ to be compatible with
seastar

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: exercise keepalive also

allow one to pass options to unittest_seastar_messenger for specifying
- verbose
- keepalive-ratio
- rounds

so we can have better coverage of crimson/messenger. and make it less
chatty unless it's asked to do so.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add keepalive support to Connection

* do not mix buffered writes and zero-copy writes.

/root/ceph/src/seastar/include/seastar/core/iostream-impl.hh:114:
seastar::future<> seastar::output_stream::write(seastar::net::packet)
[with CharType = char]: Assertion `!_end && "Mixing buffered writes and
zero-copy writes not supported yet"' failed.
Aborting on shard 0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add unittest_seastar_messenger to "make check"

as it's relatively stable at the moment. and as suggested by @cyx1231st
.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: use HAVE_FEATURE instead

so we can check the marker the bit's incarnation also.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23654 from cbodley/wip-crypto-assert-always

crypto: use ceph_assert_always for assertions

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22668 into master

* refs/pull/22668/head:
mds: reorganize and curate perf log counters
mds: show session load average in dump session
mds: allow request load average decay rate configurable via admin socket
mds: decay counter for tracking request load average
mds: add perf counters to track current open/stale sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23438 into master

* refs/pull/23438/head:
client: fix choose_target_mds for requests that do name lookup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23467 into master

* refs/pull/23467/head:
cephfs-journal-tool: enable purge_queue journal's event commands

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #23529 into master

* refs/pull/23529/head:
mon: fix idempotent FS creation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #23543 into master

* refs/pull/23543/head:
tools/cephfs: make cephfs-data-scan prints the max used ino

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23584 into master

* refs/pull/23584/head:
mds: don't modify filepath when printing
mds: hold slave request refernce when dumping MDRequestImpl

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23589 into master

* refs/pull/23589/head:
mds: more description for failed authpin
mds: cleanup CDir freezing/frozen tree check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use monotonic clock in beacon

Also update other parts of MDS which interact.

Fixes: http://tracker.ceph.com/issues/26959
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use cephfs standard configs for kclient

Specifically, qa/cephfs/begin.yaml.

Fixes: http://tracker.ceph.com/issues/26967
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/librbd: fix compiler -Wsign-compare warnings

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23315 from Devp00l/wip-clean-notification-messages

mgr/dashboard: Cleaner notifications

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23637 from noahdesu/qa-mgr-mod-path

qa: fix manager module paths

crypto: use ceph_assert_always for assertions

use ceph_assert_always() to enforce this comment in ceph_crypto.h:

This assert MUST NOT be compiled out, even on non-debug builds.

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23639 from trociny/wip-24910

librbd: potential race on image create request complete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23110 from Songweibin/wip-rbd-snap-rollback

librbd: use object map when doing snap rollback

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: journaling unable request can not be sent to remote lock owner

Fixes: http://tracker.ceph.com/issues/26939
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23609 from ErwanAliasr1/evelu-ccache

run-make-check: Showing configuration before the build

crimson/common: apply config changes also on shard.0

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Remove upstart files and references

Nautilus is not supported on any distros still running upstart so these
are redundant.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #23601 from tchaikov/wip-pybind-freebsd-clang

pybind: do not check MFLAGS

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23593 from jtlayton/wip-jlayton-26948

librados: add a rados_omap_iter_size function

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23588 from gmayyyha/ceph-df-json

mon/PGMap: command 'ceph df -f json' output add total_percent_used

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23563 from yaozongyou/mon-ping-timeout

mon/MonClient: use mon_client_ping_timeout during ping_monitor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10305 from majianpeng/memdb-perfcounter

kv/MemDB: add perfcounter.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23628 from tchaikov/wip-cmake-gtest-cleanup

cmake: link against gtest in a better way

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

core: make pthread_get_name_np work when available

FreeBSD will have a pthread_get_name_np() in libpthread
Code for that was submitted

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

cpp-btree: do not #define NDEBUG

use no_debug instead, the NDEBUG macro pollutes all source files
including this header.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/librados: do not use assert() for its side effect

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: potential race on image create request complete

Fixes: http://tracker.ceph.com/issues/24910
Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/tasks/mgr/test_module_selftest: use hello instead of status for disbled command test

Signed-off-by: Sage Weil <sage@redhat.com>

doc: move mgr/balancer to rados/operations/balancer

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: keep status, balancer always on

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #23616 into master

* refs/pull/23616/head:
doc/releases/schedule: note Jewel EOL date

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #23618 into master

* refs/pull/23618/head:
doc: updated infernalis release reference link
doc: added ref label for infernalis release notes
doc: used reST label for ceph-deploy reference
doc: added reference label for ceph-deploy page
doc: updated cross referencing locations link

Reviewed-by: Sage Weil <sage@redhat.com>

test: Fix test to use -gt instead of creating an empty file "0"

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge PR #23636 into master

* refs/pull/23636/head:
messages: fix get_opname return type

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23620 from tchaikov/wip-dpdk-cmake

cmake: add dpdk::dpdk if dpdk is built or found

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: fix error processing images which have non-user snapshots

Signed-off-by: songweibin <song.weibin@zte.com.cn>

rbd: fix error purging non-user snapshots

Fixes:
  [root@ ~]# rbd snap rm img1@snap1
  [root@ ~]# rbd snap ls img1 -a
  SNAPID NAME                                 SIZE    TIMESTAMP                NAMESPACE
       4 f2e82bd1-e2ff-4a6b-aaef-5a12a2b23a30 100 MiB Sat Aug 18 08:48:34 2018 trash (snap1)
  [root@ ~]# rbd snap purge img1
  Removing all snapshots: 0% complete...failed.
  rbd: removing snaps failed: (2) No such file or directory

Signed-off-by: songweibin <song.weibin@zte.com.cn>

qa/ceph_objectstore_tool.py: set mgr module path

Signed-off-by: Noah Watkins <nwatkins@redhat.com>

qa/standalone/ceph-helpers.sh: fix mgr module path

callers of get_python_path were not passing in a $1 parameter, so
ceph_lib was an empty string resulting in an invalid path to the built
cython modules. assume this is called from the `lib` parent directory.

pass path to the manager modules when starting ceph-mgr.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>

messages: fix get_opname return type

It cannot be a constexpr.

Introduce-by: 00b867fae2e8e912119fc47348dee78631c6f66b / #23623
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23574 from cbodley/wip-26895

rgw multisite: bucket sync transitions back to StateInit on OP_SYNCSTOP

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #23575 into master

* refs/pull/23575/head:
tools/cephfs-shell:pep8 code clean
tools/cephfs-shell:resolved removing files using pattern inside sub-directories
tools/cephfs-shell:changed output representation for the commands cwd and ls
tools/cephfs-shell:resolved removing directories using pattern inside sub-directories

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #18096 into master

* refs/pull/18096/head:
os/bluestore: factor VDO into bluestore's 'allocated' value
os/filestore: estimate omap_allocated
os/ObjectMap: add get_db() accessor
osd/OSDMap: include data, omap in 'osd df' output
os/bluestore: report omap and internal metadata usage
os/bluestore/BlueFS: drop get_fs_usage(); add get_used()
kv: add per-prefix space estimation
osd: decouple statfs update from hb peers, pg count
osd: drop useless OSDService::update_osd_stat()
osd/osd_types: separate accounting for data, omap, and metadata
osd: re-add get_num_pgs()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

tools/cephfs-shell:pep8 code clean

Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>

tools/cephfs-shell:resolved removing files using pattern inside sub-directories

Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>

tools/cephfs-shell:changed output representation for the commands cwd and ls

Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>

tools/cephfs-shell:resolved removing directories using pattern inside sub-directories

Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>

mds: simplify beacon init

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23623 from tchaikov/wip-inline-constants

messages: define HEAD_VERSION and COMPAT_VERSION inlined

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

messages: define HEAD_VERSION and COMPAT_VERSION inlined

otherwise we could have ../lib/libceph-common.so.0: undefined reference
to `MOSDPGNotify::HEAD_VERSION

when the compiler tries to be smart and pass them by reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23419 from tspmelo/fix-26856

mgr/dashboard: Add decorator to skip parameter encoding

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #23446 from tspmelo/wip-25075

mgr/dashboard: Use human readable units on the sparkline graphs

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #23388 from ricardoasmarques/wip-fix-auto-logout

mgr/dashboard: Fix redirect to login page on session lost

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

run-make-check: Showing configuration before the build

The actual code is resetting the statistics before doing the actual compilation and prints them after the build.

That is nice to understand how much the cache was used but doesn't help understanding how much it _could_ have been used.

This patch is adding a reporting (-s) when cleaning the statistics so we can estimate :
- the actual number of files in cache
- the actual size of the cache

With this two missing information, its now possible estimate if there is some miss-usage of the cache.

Signed-off-by: Erwan Velu <erwan@redhat.com>

cmake: link against gtest in a better way

* add FindGMock.cmake which allows user to use the libgtest-dev
  shipped by distro
* add GMock::{GMock,Main}, GTest::{GTest,Main} targets to be
  compatible with FindGTest.cmake and FindGMock.cmake, which
  expose the built libraries with properties adhered to
  them. so the consumer of them can import them in a better way.
* update tests to drop the commands like
  set_target_properties(foo PROPERTIES COMPILE_FLAGS ${UNITTEST_CXX_FLAGS}),
  as they are already linked against gmock and gtest.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: fix Finddpdk.cmake

* add dpdk::dpdk if dpdk is built or found, as seastar checks for
  it before adding its component libraries. if user installs libdpdk
  and builds WITH_SEASTAR=ON or WITH_DPDK=ON, cmake fails to configure
  the building system without this fix.
* add dpdk::cflags target for populating the -march=<arch> compile
  option.
* also use pkg-config for finding dpdk.
* link common_async_dpdk against dpdk::dpdk instead

please note, the reason why we can remove the "-march=native" compile
option from Finddpdk.cmake, is that the distro shipped header files
are "generic" in the sense of "-march=<ARCH>", they do not enable
the arch specific intrinsic by default, hence the source files
including them do not need specific compiler options.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not build DPDK with -march=native

-march=native causes gcc to use opcodes according to whichever CPU happens to
be installed in the build host, which can be different for every build. This
makes it impossible to achieve a reproducible build.

Also, if the build host has a very new CPU, running the resulting binaries on
older CPUs (of the same family, i.e. x86_64) could result in segmentation
fault.

References: http://tracker.ceph.com/issues/24948
Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: honour CMAKE_REQUIRED_LIBRARIES in CheckIncludeFiles

to silence cmake warning regarding to CMP0075

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: updated infernalis release reference link

Updated Infernalis reference link with reST label.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

doc: added ref label for infernalis release notes

Added reference label for Infernalis release notes.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

doc: used reST label for ceph-deploy reference

Updated ceph-deploy man page link with ref label. Fixes
404 error.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>