Merge PR #24774 into master

* refs/pull/24774/head:
cmake: update dpdk drivers/modules to accomodate dpdk submodule
spdk: update to latest v18.07

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #24666 into master

* refs/pull/24666/head:
include/types: fixed compile warning for signed/unsigned comparison
osd/PrimaryLogPG: uncommitted dup ops should respond with logged return code
osd/PrimaryLogPG: propagate error return codes on object copy_get ops
osd/PGLog: optionally record error return codes for extra_reqids
osd/osd_types: include PG log return codes in object copy data

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24688 into master

* refs/pull/24688/head:
common: make ceph_abort store same crash info as ceph_assert
global: store assert msg in global and dump to crash meta
pybind/mgr: make 'ceph crash ls' output sorted list
log: don't clear ring when dump_recent is called
ceph-crash: make clear to user that 'posted' should be directory

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24780 into master

* refs/pull/24780/head:
osd: take heartbeat_lock before checking for session
Merge PR #24725 into nautilus
qa/tasks/qemu: use unique clone directory to avoid race with workunit
mds: add missing mds_lock

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #24778 into master

* refs/pull/24778/head:
rpm: use %license macro for packaging license file

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24792 from falcon78921/wip-doc-grammar1

doc: fixed typo in man page

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24702 from wjwithagen/wjw-fix-blkdev-serial-const

common: mark BlkDev::serial() const to match with its declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fixed minor grammar error

Changed buchket to bucket

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #24785 from falcon78921/wip-docs-36605

doc: purge subcommand link broken

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: used ceph osd command ref label

Referenced purge subcommand info via ceph osd command label.
Fixes: https://tracker.ceph.com/issues/36605
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #24743 from rzarzynski/wip-osd-avoid-osdmap-refcounting

 core: avoid unnecessary refcounting of OSDMap on OSD's hot paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24657 from xiexingguo/wip-rm-device-class-fix

mon/OSDMonitor: two "ceph osd crush class rm" fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24585 into master

* refs/pull/24585/head:
doc: add developer documentation on new cephfs reclaim interfaces

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21094 from guzhongyan/dne-cleanup

src: no 'dne' acronym in user cmd output

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22633 from dongbula/add-dbstatistics-for-filestore

OSD: add impl for filestore to get dbstatistics

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23528 from jcsp/wip-osdmon-application

mon: don't commit osdmap on no-op application ops

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

osd: take heartbeat_lock before checking for session

When we open a connection, there is a short window before we attach
the session.  If a fault happens quickly, we won't get the reset, and
will persistently fail to send osd pings.

Move the lock up to avoid this.  Note that we should rarely really see
connections without sessions here anyway (except when this specific
race happens), so this should have no negative impact (by taking the lock
when we weren't before).

Fixes: http://tracker.ceph.com/issues/36602
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #24504 from ErwanAliasr1/evelu-ceph-volume-choose_disk

Additional work on ceph-volume to add some choose_disk capabilities

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #24703 into master

* refs/pull/24703/head:
common: prefer std::size to ARRAY_SIZE macro

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24712 into master

* refs/pull/24712/head:
mgr/DaemonServer: "osd safe-to-destroy" - more verbose output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24713 into master

* refs/pull/24713/head:
mon: drop repeated 'goodchars' and add osd crush ls testcase

Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #24724 into master

* refs/pull/24724/head:
test/librados/aio: remove unused callbacks
test/librados/aio: wait for all completions properly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24736 into master

* refs/pull/24736/head:
os/tests: should read size_t options using get_val<Option::size_t>()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24767 from votdev/issue_36581

mgr/dashboard/qa: Fix various vstart_runner.py issues

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>

rpm: use %license macro for packaging license file

see https://rpm-guide.readthedocs.io/en/latest/rpm-guide.html#rpm-macros

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: update dpdk drivers/modules to accomodate dpdk submodule

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24584 from wido/docs-zabbix-template

doc: Update mgr/zabbix plugin documentation with link to Zabbix template

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #24606 from p-na/osd-actions-fe

mgr/dashboard: Add support for managing individual OSD settings/characteristics in the frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

doc: Update mgr/zabbix plugin documentation with link to Zabbix template

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #24297 from ukernel/wip-36192

osdc: reduce ObjectCacher's memory fragments

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

spdk: update to latest v18.07

Fixes: http://tracker.ceph.com/issues/36550
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24756 from tspmelo/wip-contribs

.githubmap, .mailmap, .organizationmap: update contributors

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #24727 from zmc/wip-dashboard-gzip

mgr/dashboard: Enable gzip compression

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

.githubmap, .mailmap, .organizationmap: update contributors

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard/qa: CephfsTest - admin_socket() got an unexpected keyword argument 'timeout'

Adapt method arguments of LocalRemote::run() according to teuthology.orchestra.run.run() (see https://github.com/ceph/teuthology/blob/master/teuthology/orchestra/run.py#L364) to be able to run QA tests locally in a vstart cluster.

Fixes: http://tracker.ceph.com/issues/36581
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #24720 from gauravsitlani/master

doc: added rbd default features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: add developer documentation on new cephfs reclaim interfaces

Tracker: http://tracker.ceph.com/issues/36395
Signed-off-by: Jeff Layton <jlayton@redhat.com>

doc: added rbd default features

Fixes: https://tracker.ceph.com/issues/15000
Added RBD default features description

Signed-off-by: Gaurav Sitlani <gauravsitlani@riseup.net>

Merge PR #24716 into master

* refs/pull/24716/head:
test/log: drop redundant test case
common/StackStringStream: don't reserve before every insert

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24283 into master

* refs/pull/24283/head:
client: support for exporting multiple subdirectories in faked mode

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #24050 into master

* refs/pull/24050/head:
Revert "mds: adjust export size after renaming directory into freezing subtree"
mds: cleanup nested auth pins tracking

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24700 from cbodley/wip-27221

rgw: support server-side encryption when SSL is terminated in a proxy

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #24721 from cbodley/wip-librados-mono-timeout

librados: fix unitialized timeout in wait_for_osdmap

Reviewed-by: Mohamad Gebai <mgebai@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24680 from cbodley/wip-36537

rgw: fix vector index out of range in RGWReadDataSyncRecoveringShardsCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #24677 from cbodley/wip-rgw-beast-derr

rgw: beast frontend logs socket errors at level 4

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #24731 into master

* refs/pull/24731/head:
rados: fix admin/build-doc warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24641 from ricardoasmarques/fix-cloning-message

mgr/dashboard: Fix long running RBD cloning / copying message

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24587 from ceph/wip-rm36363

ceph-volume: adds a --prepare flag to `lvm batch`

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24750 from iotcg/master

doc: give pool name if default pool rbd is not created

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: give pool name if default pool rbd is not created

unless creating the default pool named to be "rbd",
it will fail without pointing the pool name when
using rbd command.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

ceph-volume: update man page for batch --prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

mgr/dashboard: Fix long running RBD cloning / copying message

Fixes: https://tracker.ceph.com/issues/36484
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #24693 from idryomov/wip-is-queued

msg/async: fix is_queued() semantics

Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add support for managing individual OSD settings/characteristics in the frontend

Fixes: http://tracker.ceph.com/issues/35448
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #24739 from tspmelo/wip-py3-itervalues

ceph.in: Add support for python 3

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23962 from aclamk/fio-fix-compilation

objectstore/test/fio: Fixed fio compilation when tcmalloc is used.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24725 into nautilus

* refs/pull/24725/head:
mds: add missing mds_lock

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Enable gzip compression

This is related to http://tracker.ceph.com/issues/36453. It is far from
a complete solution, but seems like a positive move.

I tested this change by first disabling my browser cache, and then used
the /docs endpoint to query /api/dashboard/health. Before compression:
  Content-Length: 60748
  Time: 615ms
After:
  Content-Length: 7505
  Time: 92ms

Then, I logged into the dashboard as normal and reloaded the page once I
was in. Some values for the reload operation before compression:
  Total page load time: 58.48s
  vendor.js Content-Length: 6486025
  vendor.js time: 48.09s
After:
  Total page load time: 14.55s
  vendor.js Content-Length: 1143178
  vendor.js time: 4.50s

Signed-off-by: Zack Cerza <zack@redhat.com>

DashboardTestCase: add assertHeaders()

Signed-off-by: Zack Cerza <zack@redhat.com>

LocalCephManager.admin_socket: add timeout kwarg

This fixes "TypeError: admin_socket() got an unexpected keyword argument
'timeout'". The value is never used.

Signed-off-by: Zack Cerza <zack@redhat.com>

core: avoid get_epoch()-driven refcounting on OSDMap in ReplicatedBackend.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

core: avoid get_epoch()-driven refcounting on OSDMap in PrimaryLogPG.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph.in: Add support for python 3

This will allow the condition to work in both python 2 and 3 environments.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24738 from alfredodeza/wip-rm36492

ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #24665 into master

* refs/pull/24665/head:
common: remove unused ioprio thread/wq infrastructure

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge PR #24701 into master

* refs/pull/24701/head:
os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #24707 into master

* refs/pull/24707/head:
build/ops: drop sgdisk build/runtime dependencies

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge PR #24687 into master

* refs/pull/24687/head:
src/test: fix unordered manifest-unset op

Reviewed-by: Sage Weil <sage@redhat.com>

rados: fix admin/build-doc warnings

Fixed admin/build-doc warnings in librados.h
and rados_types.h

Signed-off-by: Jos Collin <jcollin@redhat.com>

ceph-volume tests.api update monkeypatching to use **kw

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests allow **kw when monkeypatching

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume api.lvm: do not spit out errors on the terminal from LVM

Signed-off-by: Alfredo Deza <adeza@redhat.com>

qa/tasks/qemu: use unique clone directory to avoid race with workunit

If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5d56014c61b107dcb5d05c2221c2e844324f304c)

os/tests: should read size_t options using get_val<Option::size_t>()

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24722 from zmc/rbd-redirect

mgr/dashboard: Redirect /block to /block/rbd

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Simplify frontend test helper

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: TableActionsComponent cleanup

Rename `buttonCondition` to `canBePrimary`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Rename `DeletionModalComponent` to `CriticalConfirmationModalComponent`

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Cleanup OSD components

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

common: prefer std::size to ARRAY_SIZE macro

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

Merge pull request #19964 from runsisi/wip-build

cmake: move ceph-osdomap-tool, ceph-monstore-tool out of ceph-test

Reviewed-by: Kefu Chai <kchai@redhat.com>

librados: fix unitialized timeout in wait_for_osdmap

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24540 into master

* refs/pull/24540/head:
mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add missing mds_lock

Fixes: http://tracker.ceph.com/issues/36573
Introduced-by: ecbd4a8aa8e6c1c72af4e0be15e0340629bfdc3a
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24709 into master

* refs/pull/24709/head:
PendingReleaseNotes: note about cephfs client state reclaim

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24466 into master

* refs/pull/24466/head:
common: assert if buffer advance length overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24529 into master

* refs/pull/24529/head:
mds: cleanup command descriptions

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #24640 into master

* refs/pull/24640/head:
mds: remove MonClient reconnect

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24533 into master

* refs/pull/24533/head:
qa: add timeouts for remote ops for client mounts

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #24679 from tspmelo/wip-36528

mgr/dashboard: Fix HttpClient Module imports in unit tests

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #24632 from Devp00l/wip-issue-36468

mgr/dashboard: Handle class objects as regular objects in KV-table

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

test/librados/aio: remove unused callbacks

That callbacks are called is covered by many other tests.
These particular m_safe and m_complete fields were unused.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

test/librados/aio: wait for all completions properly

Using global semaphores was error-prone and unnecessary. Unbalanced
usage resulted in a race like:

aio_write object foo -> dropped on full OSD
aio_read object foo -> -ENOENT

The intent of the test was to wait for the write to complete, so
do that explicitly for each aio test and remove the semaphores.

Fixes: http://tracker.ceph.com/issues/24587
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24674 from sileht/master

ceph-volume: extract flake8 config

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Redirect /block to /block/rbd

Fixes: https://tracker.ceph.com/issues/36401
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #23242 from linuxbox2/wip-rgw-inflight-ops

rgw: set default objecter_inflight_ops = 24576

common: Fix function const, when not all serial() are const.

And then Clang complains:
```
/home/jenkins/workspace/ceph-master/src/common/blkdev.cc:700:13: error: out-of-line definition of 'serial' does not match any declaration in 'Bl
kDev'
int BlkDev::serial(char *serial, size_t max)
            ^~~~~~
/home/jenkins/workspace/ceph-master/src/common/blkdev.h:50:7: note: member declaration does not match because it is const qualified
  int serial(char *serial, size_t max) const;
      ^                                ~~~~~
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

common: remove unused ioprio thread/wq infrastructure

This is all now obsolete, with no remaining users.

Move ceph_gettid into Thread.{cc,h}

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: do not remove device class still referenced by ec-profiles

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: make "ceph osd crush class rm" idempotent

Removing a non-existent device class should be generally okay.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>