ceph-volume: extract flake8 config

It's preferable to use [flake8] section to configure flake8.

So external tools/editor can read the configuration.

Signed-off-by: Mehdi Abaakouk <sileht@sileht.net>

Merge pull request #24314 from rhcs-dashboard/dashboards

mgr/dashboard: Grafana dashboard updates and additions

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #24628 from Devp00l/wip-issue-36445

mgr/dashboard: Add missing test requirement "werkzeug"

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #24669 from falcon78921/wip-docs-36527

doc: add spacing to subcommand references

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19386 from ganeshmaharaj/qat-ceph-crypto

crypto: QAT based Encryption for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24618 from CharlotteRose/wip-number-eviction-events

doc: numbered eviction situations

Reviewed-by: Jos Collin <jcollin@redhat.com>

crypto: Merge compilation of crypto accelerators under a single cmake.

- Control ISA-L and QAT under a crypto makefile
- Allow conditional compile of QAT.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

crypto: QAT Wrapper for Ceph Encryption

- Crypto plugin created and currently handles only AES_256_CBC
- Contents
-- Poller thread for QAT HW
-- Contiguous memory allocator using QAT's usdm driver
-- Do in-place crypto operation reducing mem usage.
-- Enable multi-QAT instance to allow parallel operations.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>

doc: add spacing to subcommand references

Added appropriate spacing for subcommand
code references.

Fixes: https://tracker.ceph.com/issues/36527
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge pull request #24531 from alfredodeza/wip-rm36386

ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24453 from cbodley/wip-rgw-putobjprocessor

rgw: refactor PutObjProcessor stack

Merge pull request #23304 from smithfarm/wip-23776

 tools: ceph-authtool: report correct number of caps when creating keyring

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: numbered eviction situations

Signed-off-by: Scoots Hamilton <scoots@redhat.com>

Merge pull request #19571 from Fusl/patch-1

doc: doc/radosgw/s3.rst: Adding AWS S3 `Storage Class` as `Not Supported`

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24660 from tchaikov/wip-crimson-cleanup

crimson: crimson/mon: remove timeout support from mon::Client::authenticate()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24626 from jtlayton/wip-mgr-urlparse

mgr/rook: fix urljoin import

Merge pull request #24604 from yangdongsheng/token_fix

common/Throttle: TokenBucketThrottle: use reference to m_blockers.front()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24653 from mynaramana/claydoc

doc: erasure-code-clay fixes typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22162 from liuzhanhao/master

doc: Update cpp.rst to accommodate the new APIs in libs3

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: erasure-code-clay fixes typos

Signed-off-by: Myna <mynaramana@gmail.com>

tests/ceph-authtool: add reproducer for "with 0 caps" issue

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: ceph-authtool: report correct number of caps when creating

Fixes: https://tracker.ceph.com/issues/23776
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #24503 into master

* refs/pull/24503/head:
qa: increase timeout for cleanup

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #24455 into master

* refs/pull/24455/head:
qa: use timeout for fs asok operations

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #24615 into master

* refs/pull/24615/head:
cmake: mark gcc 7 as minimum C++ compiler (due to std++17)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24617 from p-na/fix-python3-issue

mgr/dashboard: Fix Python3 issue

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

crimson/mon: remove timeout support from mon::Client::authenticate()

as the timeout parameter is not used by OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: reuse common_mountcephfs_objs in crimson-common

so we won't built them twice.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #24102 into master

* refs/pull/24102/head:
global: fix err_to_stderr can't work when it was configured to false
osd: replace cout/derr with dout(0) to avoid writing trivial logs to syslog file

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24236 into master

* refs/pull/24236/head:
osd: Log error and abort if fails to pick address

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #24359 into master

* refs/pull/24359/head:
qa/tests: update ansible version to 2.6 for master branch testing.
qa/tests: use lvm as default for ceph-ansible testing, this should also work with raw devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PR #24603 into master

* refs/pull/24603/head:
crush: get "ceph osd crush class create/rm" back

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24613 into master

* refs/pull/24613/head:
OSD: fix a typo err when increase recvoery bytes statistics

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24586 from mynaramana/claydoc

erasure-code: fixes alignment issue when clay code is used with jerasure, cauchy_orig

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/rook: fix urljoin import

This is the correct syntax to ensure that we get urljoin, no matter
the python version.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #24608 from cyx1231st/wip-seastar-msgr-accepting

crimson/net: implement accepting/connecting states

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/net: implement accepting/connecting states

- introduce the new accepting/connecting connection states.
- return ConnectionRef immediately when connect().
- manage the ownership of the accepting connections.
- manage the ownership of the registered connections.
- encapsulate a Socket class because it is not created when constructing
  a SocketConnection, and allow it to be replaced in the future.
- refactor related interfaces.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge PR #24624 into master

* refs/pull/24624/head:
client: Mutex::Locker -> std::lock_guard
mds: Mutex::Locker -> std::lock_guard

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

erasure-code: fixing bugs in clay-code plugin

* Fixes alignment issue while using clay code with jerasure, cauchy_orig
* Fixes ceph_erasure_code_non_regression to adapt to new decode

Signed-off-by: Myna <mynaramana@gmail.com>

mgr/dashboard: Fix Python3 issue

Which results in a 500 error when trying to access the `Performance
Counter` tab on the OSD list.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #24629 from cyx1231st/wip-seastar-fix

crimson/net: fix compile errors in test_alien_echo.cc

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/net: fix compile errors

test_alien_echo.cc is using common_init_finish, but it is not available
if defined WITH_SEASTAR.

Dispatcher::ms_verify_authorizer() is removed by PR#24095.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge pull request #24607 from p-na/fix-osd-list-status-labels

mgr/dashboard: Fix spaces around status labels on OSD list

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24614 from s0nea/wip-dashboard-configs-textarea-vertical-resize

mgr/dashboard: configs textarea disallow horizontal resize

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #24631 from Devp00l/wip-issue-36466

mgr/dashboard: Add left padding to helper icon

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add left padding to helper icon

Fixes: https://tracker.ceph.com/issues/36466
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add missing test requirement "werkzeug"

Fixes: https://tracker.ceph.com/issues/36445
Signed-off-by: Stephan Müller <smueller@suse.com>

doc: erasure-code cleanup of clay plugin documentation

Signed-off-by: Myna <mynaramana@gmail.com>

TokenBucketThrottle: use reference to m_blockers.front() and then update it

As we need to update the blocker below, then we need to use reference to front object,
rather than a new object of blocker.

Fixes: http://tracker.ceph.com/issues/36475
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge PR #24578 into master

* refs/pull/24578/head:
pybind/ceph_argparse.py: do not create file for validating CephFilepath

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #24565 into master

* refs/pull/24565/head:
mgr: python 3 compat fixes

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #24558 into master

* refs/pull/24558/head:
pybind/mgr: Fix Python 3 imports in diskprediction & insights

Reviewed-by: Noah Watkins <nwatkins@redhat.com>

Merge PR #24528 into master

* refs/pull/24528/head:
common/condition_variable_debug: fix wait hooks
common/mutex_debug: remove no-op before/after hooks
common/mutex_debug: do lockdep post-lock step in caller
os/bluestore: {Mutex,Cond} -> ceph::{mutex,condition_variable}
os/bluestore: std::recursive_mutex -> ceph::recursive_mutex
os/bluestore: re-add is_locked assert
os/bluestore: std::{mutex,condition_variable} -> ceph::{mutex,condition_variable}
os/bluestore: use deduction for lock_guard<>, unique_lock<>

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>

Merge PR #24151 into master

* refs/pull/24151/head:
mgr/devicehealth: use is_valid_daemon_name helper
mgr/devicehealth: generalize to mon and osd daemons
mon: implement 'smart [devid]' tell command
mgr: parse mon metadata properly
mon: report device id used by mon
common/blkdev: add get_device_by_path
common/blkdev: migrate block_device_run_smartctl from OSD.cc

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #24566 into master

* refs/pull/24566/head:
osd,mon: keep last_epoch_started along with last_epoch_clean premerge

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

client: Mutex::Locker -> std::lock_guard

Signed-off-by: Sage Weil <sage@redhat.com>

mds: Mutex::Locker -> std::lock_guard

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24598 into master

* refs/pull/24598/head:
.github/stale.yml: configure probot/stale to automatically close stale issues

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #24619 from tchaikov/wip-crimson-mon-client-better-encapsulation

crimson/mon: move mon::Connection into .cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crimson/mon: move mon::Connection into .cc

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23849 from tchaikov/wip-crimson-monc

crimson: add MonClient

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: remove old PutObjProcessor stack

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: copy_obj uses AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove RGWPutObj::select_processor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWBulkUploadOp uses AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: PostObj uses AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fetch_remote_obj() uses AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: prepare RGWRadosPutObj for fetch_remote_obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWObjectCtxImpl::invalidate takes const ref

also removes the default implementation, which is duplicated in the
specialization for rgw_obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rgw_file uses AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: PutObj::execute() uses new ObjectProcessors

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: compression filter uses rgw::putobj::Pipe

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: encryption filter uses rgw::putobj::Pipe

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: temporarily disable putobj filters

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: read multipart params in RGWPutObj_ObjStore_S3::get_params

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: PutObj avoids some static_casts to RGWObjectCtx

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove gotos from PutObj

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: PutObj uses unique_ptr to manage PutObjProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: PutObj uses scope_guard to update perf counter on return

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::putobj::MultipartObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::putobj::AtomicObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add const to some RGWBucketInfo helpers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::putobj::ManifestObjectProcessor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add const to some manifest generator functions

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::putobj::RadosWriter adapter

implements the DataProcessor interface by writing its buffers with Aio,
and tracks the set of successful writes so they can be deleted on
failure/cancelation

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #24505 into master

* refs/pull/24505/head:
mds: wait shorter intervals if beacon not sent

Reviewed-by: Zheng Yan <zyan@redhat.com>

cmake: mark gcc 7 as minimum C++ compiler (due to std++17)

Ceph requires C++17 support from compiler. This means gcc 7.x being
minimal version supported.

This also allows to fail quick on Debian 'stretch' with it's gcc 6.3
compiler.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Merge pull request #24373 from mogeb/build-cls-rbd

osd: add required cls_* libraries as dependencies of osd

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

mds: wait shorter intervals if beacon not sent

MDS beacon upkeep always waits mds_beacon_interval seconds even when laggy.
Check more frequently for when we stop being laggy to reduce likelihood that
the MDS is removed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

global: fix err_to_stderr can't work when it was configured to false

function `set_stderr_level(-1, -1)` set m_stderr_log and m_stderr_crash to -1,
regardless of whether `err_to_stderr` is set to false or not, so logs will be
 always written to stderr. fix it as the same way as handle_conf_change does.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

osd: replace cout/derr with dout(0) to avoid writing trivial logs to syslog file

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge PR #24095 into master

* refs/pull/24095/head:
osd: do not authenticate heartbeat connections until nautilus
msg,osd: enable unauthenticated Dispatcher for pre-nautilus OSD compat
osd: add missing space in heartbeat debug output
mgr/DaemonServer: add missing return
msg/async: drop verify_authorizer wrapper
osd: authenticate ping sessions
msg/simple: remove verify_authorizer wrapper
msg: remove unused ms_verify_authorizer
msg/async: remove get_authorizer wrapper
msg/simple: remove get_authorizer wrapper
msg/Messenger: pull authenticator validation into Messenger
msg/Messenger: uninline ms_deliver_verify_authorizer
mgr/DaemonServer: expose keyring for authenticator verification
mon: expose keyring for msgr1 authentication
mds: expose keyring for authenticater verification
osd: expose keyring for authenticater verification
msg/Dispatcher: add ms_get_auth1_authorizer_keystore
mon: fix ref cycle breakage in handle_forward
mon: use MonOpRequest get_session() instead of PaxosServiceMessage's
mon: get session from MonOpRequest in handle_command
messages/MForward: drop unused ctor
mon: use ms_handle_authentication to parse caps
mon: kill Session::global_id and use Connection member instead
mgr/DaemonServer: move session setup into ms_handle_authentication
mds: move session setup into ms_handle_authentication
osd: move session setup into ms_handle_authentication
msg: new ms_handle_authentication, add fields to Connection

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #24579 into master

* refs/pull/24579/head:
qa/osd: fixup osd-rep-recov-eio.sh fails to parse pg dump

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24597 from batrick/i36450

qa: fix run call args

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #23983 from theanalyst/vstart-rgw-asok

vstart: set admin socket for RGW in conf

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: configs textarea disallow horizontal resize

The textarea allows horizontal and vertical resize by default. Only the
vertical resize is appropriate for this form.

Fixes: http://tracker.ceph.com/issues/36452
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Fix spaces around status labels on OSD list

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

OSD: fix a typo err when increase recvoery bytes statistics

Signed-off-by: kungf <yang.wang@easystack.cn>

Merge pull request #24576 from cyx1231st/wip-seastar-msgr-refactor

crimson/net: seastar-msgr refactoring

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24255 from gregsfortytwo/wip-crush-doc

doc: explain 'firstn v indep' in the CRUSH docs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>