doc/rados/operations/health-checks: Add LARGE_OMAP_OBJECTS

Document LARGE_OMAP_OBJECTS health check

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #25223 from smithfarm/wip-rhel-8-py3

build/ops: rpm: RHEL 8 needs Python 3 build

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #25215 from majianpeng/test_shared_cache_memory_leak

test_shared_cache: fix memory leak.

Merge PR #24798 into master

* refs/pull/24798/head:
fix python collections module warning for v3.7 and above

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #25054 into master

* refs/pull/25054/head:
test/objectstore: enable fsck when needed.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25247 from wjwithagen/wjw-fix-linker

do_freebsd.sh: FreeBSD building needs the llvm linker

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25194 from dillaman/wip-36650

librbd: extend API to include parent/child namespaces and image ids

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24831 from p-na/wip-pna-cleanup-fe

mgr/dashboard: Cleanup

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #24732 into master

* refs/pull/24732/head:
os/bluestore: fix missing assert->ceph_assert replacement.
os/bluestore: add bluefs perf counters to track DB/WAL/Slow volume

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #24499 into master

* refs/pull/24499/head:
common: reorder bl::iter to reduce its size from 40 to 32 on x86_64.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #25248 into master

* refs/pull/25248/head:
mgr: fix Clang warning on superfluous brackets

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24949 into master

* refs/pull/24949/head:
include/ceph_features: note kernel versions some features appear

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25225 from Aran85/sangfor-aff

mailmap,organization: Update sangfor affiliation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25150 from majianpeng/shared_cache_race_bug

common/shared_cache: fix racing issues

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25165 from hsiang41/mgr/diskprediction_cloud_global_config

mgr: Diskprediction cloud activate when config changes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25105 from tchaikov/wip-ceph-mutex-common

common: Mutex -> ceph::mutex

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mailmap,orgnization: Update sangfor affiliation

Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>

Merge pull request #23834 from DikshaGodbole/wip-feature-24571

mgr/dashboard: Move Cluster/Audit logs from front page to dedicated Logs page

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #25239 from cbodley/wip-rgw-rados-aio-cleanup

rgw: remove unused aio helper functions

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #17526 into master

* refs/pull/17526/head:
qa/tasks/ceph_manager: avoid test_map_discontinuity stall with too few up osds

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #21444 into master

* refs/pull/21444/head:
qa: Replace 'ceph' with cluster name in restart()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #25185 into master

* refs/pull/25185/head:
auth/AuthSessionHandler: no handler if no session key

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mgr: fix Clang warning on superfluous brackets

```
/home/jenkins/workspace/ceph-master/src/mgr/OSDPerfMetricTypes.h:105:19: warning: braces around scalar initializer [-Wbraced-scalar-init]
        d.regex = {d.regex_str.c_str()};
                  ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

build: FreeBSD building needs the llvm linker

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

rbd: avoid usage of newly deprecated API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: expose optional namespace in parent/child image output

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: expand API to include namespaces and image ids

The get parent and list children methods now return a common
struct that includes the full pool details (including the
optional namespace). Additionally, a new image list method has
been added which includes both the name and id (of v2 images).

Fixes: http://tracker.ceph.com/issues/36650
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25206 from dillaman/wip-36622

pybind/rbd: add namespace helper API methods

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Added new logs component

Added new Logs component and created a cluster->logs menu item. Moved the logs from the dashboard home page to the dedicated logs page.Additional comments resolved.Dropped Logs from health component.

Fixes: https://tracker.ceph.com/issues/24571
Signed-off-by: Diksha Godbole <diksha.godbole@gmail.com>

Merge pull request #25221 from tspmelo/wip-37354

mgr/dashboard: Fix test_remove_not_expired_trash qa test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #24757 from votdev/issue_36480

mgr/dashboard: Can't handle user editing when tenants are specified

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr: Diskprediction_cloud action when config changes

Trigger diskprediction cloud action when receive notify configuration changed.

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>

Merge pull request #25186 from tchaikov/wip-vive-le-make-install

cmake: do not pass unnecessary param to setup.py

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #25197 from iridescent-rsy/fix-typo

rbd-mirror,common: fix typos in logging messages and comments

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25202 from trociny/wip-doc-rbd-qos

doc: describe RBD QoS settings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>

pybind/rbd: fix python3 runtime compatibility issues

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

build/ops: rpm: RHEL 8 needs Python 3 build

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25182 from jan--f/mgr-prometheus-doc-mention-instance-label

doc/mgr/prometheus: readd section about custom instance labels

Merge pull request #25119 from sebastian-philipp/doc-orchestrator-cli-spec

doc/orchestrator: Integrate CLI specification into the documentation

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

mgr/dashboard: Fix test_remove_not_expired_trash qa test

Fixes: http://tracker.ceph.com/issues/37354
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #25144 from rhcs-dashboard/wip-37295-master

mgr/dashboard: Show pool dropdown for block-mgr

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #25159 from tspmelo/wip-i18n-doc

mgr/dashboard: Update I18N documentation

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #25181 from smithfarm/wip-37301

build/ops: rpm: stop install-deps.sh clobbering spec file Python build setting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #25205 from smithfarm/wip-37330

build/ops: do_cmake.sh: automate py3 build options for certain distros

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25211 from badone/wip-selinux-cr-repo-issue

install-deps.sh: Remove CR repo

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24821 from k0ste/prometheus_metadata

mgr: prometheus: added bluestore db and wal/journal devices to ceph_disk_occupation metric.

test_shared_cache: fix memory leak.

Based on SharedLRU::add,  If the key already existed, you are
responsible for deleting the new value you tried to insert.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #25200 from tchaikov/wip-silence-Wunused

erasure-code,test: silence -Wunused-variable warnings

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25199 from tchaikov/wip-mgr-silence-py2-warning

mgr: silence GCC warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25204 from cbodley/wip-cmake-without-rbd-cephfs

cmake: vstart target can build WITH_CEPHFS/RBD/MGR=OFF

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/Throttle: update BackoffThrottle::get() to work with unsigned delay

* in fa3acba3, we switched the delay's type from
  std::chrono::duration<double> to ceph::timespan, the former's Rep is a
  signed type, while the latter is unsigned. so we need to update the
  algorithm in get() to accomodate this change.
* use mono_clock instead of real_clock for better performance
* check for 0 timespan using count() == 0, instead of comparing it with
  a temporary object created using ceph::timespan(0) for better
  performance

Signed-off-by: Kefu Chai <kchai@redhat.com>

common,osd: Mutex -> ceph::mutex

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25114 from trociny/wip-osd-perf-query-generic

 mgr: generalize osd perf query and make counters accessible from modules

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: remove unused aio helper functions

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: fix a typo.

Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>

Merge pull request #25209 from runsisi/wip-fix-missing-prefix

test/librbd: metadata key for config should be prefixed with conf_

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24936 from mikechristie/igw-selinux

selinux: add support for ceph iscsi

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24965 from yangdongsheng/token_fix

common: complete all throttle blockers when we set average or max to 0

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25055 from runsisi/wip-fix-resize

librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25152 from Songweibin/wip-fix-get-key

 rbd: fix error parse arg when getting key

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

install-deps.sh: Remove CR repo

Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.

Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

test/librbd: metadata key for config should be prefixed with conf_

Signed-off-by: runsisi <luo.runbing@zte.com.cn>

Merge pull request #24452 from noahdesu/doc-edit-report

doc: edit on github

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/orchestrator: Integrate CLI specification into the documentation

Also: Make it clear that the orchestrator is WIP

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

build/ops: do_cmake.sh: automate py3 build options for certain distros

Before this patch, run-make-check.sh was setting the cmake options for
py3 build based only on the absence of a python2 binary in the system.
This was not sufficiently deterministic, and it didn't help folks who
run do_cmake.sh directly.

With this patch, we explicitly force the py3 build in do_cmake.sh, for
those OS types/versions we know need it.

Fixes: https://tracker.ceph.com/issues/37330
Signed-off-by: Nathan Cutler <ncutler@suse.com>

pybind/rbd: add namespace helper API methods

Fixes: http://tracker.ceph.com/issues/36622
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Update I18N documentation

Add a Suggestions section.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #25157 from tspmelo/wip-e2e-stop

mgr/dashboard: Fix run-frontend-e2e-tests.sh

Reviewed-by: Laura Paduano <lpaduano@suse.com>

cmake: vstart-base only depends on ceph-mgr if WITH_MGR

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: dont depend on cython_rbd if WITH_RBD=OFF

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: dont build ceph-mds if WITH_CEPHFS=OFF

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25155 from rhcs-dashboard/37283-mgr-info-card-improvements

mgr/dashboard: Status info cards' improvements

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc: describe RBD QoS settings

Signed-off-by: Mykola Golub <mgolub@suse.com>

erasure-code,test: silence -Wunused-variable warnings

GCC 7.3 does not support [[maybe_unused]] very well, so it emits
-Wunused-variable warnings even if the variables are marked with
maybe_unused. moreover, the C++17 standard does not forbid these
warnings:

> For an entity marked maybe_unused, implementations are encouraged not
to emit a warning that the entity is unused, or that the entity is used
despite the presence of the attribute.

see also n4659, dcl.attr.unused

so, in this change, the warnings are silenced manually.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Status info cards' improvements
- MGR info card:
  Show amount of daemons active instead of active mgr name.
  Daemons active text line: show mgr active name when 'on mouse over'.
- OSD/MDS info cards (for consistency):
  The same text appearance as MGR card.
- Hosts/Object Gateway/iSCSI cards: added 'total' after amount for clarification.
- Some CSS refactor.

Fixes: https://tracker.ceph.com/issues/37283
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #24610 from SUSE/wip-orchestrator-deepsea-redux

mgr/deepsea: DeepSea orchestrator module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>

mgr/deepsea: DeepSea orchestrator module

This module provides integration between Ceph's orchestrator framework
and DeepSea, a Ceph deployment and management system built upon Salt.

Included so far are implementations of get_inventory() and
describe_service().  This allows `ceph orchestrator device ls`, `ceph
orchestrator service ls` and `ceph orchestrator service status` to
operate correctly.

To test, try:

  # ceph mgr module enable orchestrator_cli
  # ceph mgr module enable deepsea
  # ceph orchestrator set backend deepsea
  # ceph deepsea config-set salt_api_url <salt api url>
  # ceph deepsea config-set salt_api_username <salt api username>
  # ceph deepsea config-set salt_api_password <salt api shared secret>
  # ceph orchestrator device ls
  # ceph orchestrator service ls

Signed-off-by: Tim Serong <tserong@suse.com>

common/mutex_debug: disable lockdep for recursive_mutex

without this change, when using recursive_mutex on Debug build, we will
have

 1: (ceph::mutex_debug_detail::mutex_debugging_base::_will_lock(bool)+0x4a)
[0x7fb751e36534]
 2: (ceph::mutex_debug_detail::mutex_debug_impl<true>::lock(bool)+0x3b)
[0x7fb751ca33b5]
 3: (std::lock_guard<ceph::mutex_debug_detail::mutex_debug_impl<true>
>::lock_guard(ceph::mutex_debug_detail::mutex_debug_impl<true>&)+0x2f)
[0x7fb751ca2e79]
 4: (ConfigProxy::get_val(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, char**, int)
const+0x41) [0x7fb751ce65fb]
 5: (TracepointProvider::verify_config(ConfigProxy const&)+0xbf)
[0x7fb751ce63f3]
 6: (TracepointProvider::handle_conf_change(ConfigProxy const&,
std::set<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::less<std::__cx
x11::basic_string<char, std::char_traits<char>, std::allocator<char> >
>, std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > > >
const&)+0x93) [0x7fb751ce62eb]
 7: (void ObserverMgr<ceph::md_config_obs_impl<ConfigProxy>
>::call_all_observers<ConfigProxy>(ConfigProxy const&)+0x1ab)
[0x7fb751d4f1e1]
 8: (ConfigProxy::call_all_observers()+0x50) [0x7fb751d4c570]
 9: (CephContext::start_service_thread()+0x121) [0x7fb751d47eff]

because we have following sequence:

1. acquire ConfigProxy::lock
2. acquire TracepointProvider::m_lock
3. acquire ConfigProxy::lock

which annoys lockdep, because it's a cyclic graph.

after this change,

* we will not ignore lockdep when locking/unlocking
  recursive_mutex. this basically disables lockdep checks for
  mutex_recursive_debug, but it still has nlock checks.
* also, enable_lockdep() is extracted. as we are repeating it
  for multiple times in this header file.

see 41522699, and see http://tracker.ceph.com/issues/12614

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: silence GCC warning

the signature of PyObject_CallMethod() is different in python2 and
python3:

in python2: it is

PyObject* PyObject_CallMethod(PyObject *o, char *method, char *format,
...)

while in python3, it is

PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const
char *format, ...)

so, if we compile mgr with python2, we will have following warning:

warning: ISO C++ forbids converting a string constant to ‘char*’
[-Wwrite-strings]
        (char*)NULL);
                   ^

it'd be simpler if we just const_cast<> the method name string, to
silence the warning just like other places we call
PyObject_CallMethod().

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonMap: include common/Clock.h

for using ceph_clock_now()

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/perf_counters: extract m_name(name_c_str()) out

it was wrongly duplicated in 8bf92fb6

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/shardptr_registry: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

common/simple_cache: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

common/random_cache: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

common/WorkQueue: no WorkQueue or ThreadPool on seastar

At least for now!

Signed-off-by: Sage Weil <sage@redhat.com>

common/RefCountedObject: no cond-related types

These aren't needed for seastar, and they don't build anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

common/Throttle: ceph::mutex -> std::mutex

These are inner locks; no need for lockdep here, and throttles are
in hot paths.

Signed-off-by: Sage Weil <sage@redhat.com>

common/convenience: remove lock helpers

These aren't so useful now that we're in an C++17 world.

Signed-off-by: Sage Weil <sage@redhat.com>

common/Throttle: std::mutex -> ceph::mutex, drop convenience.h-isms

- use ceph::mutex where we can (we keep Mutex for TokenBucketThrottle
  since it takes Mutex*)
- drop the UNIQUE_LOCK_T weirdness
- uniquely_lock -> std::lock_guard or std::unique_lock

Signed-off-by: Sage Weil <sage@redhat.com>

common/Throttle: use ceph::timespan

This will play nice with ceph::condition_variable_debug.

Signed-off-by: Sage Weil <sage@redhat.com>

global/signal_handler: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

osd: guardedly_lock -> std::lock_guard

Signed-off-by: Sage Weil <sage@redhat.com>

common/ContextCompletion: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

common/WorkQueue: Mutex -> ceph::mutex

Lots of #include push-downs as a result.

Signed-off-by: Sage Weil <sage@redhat.com>

common/WorkQueue: drop unused get_lock()

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: push down Mutex #include

Signed-off-by: Sage Weil <sage@redhat.com>

common/Semaphore: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

common/Readahead: Mutex -> ceph::mutex

Still need to kill C_SaferCond later.

Signed-off-by: Sage Weil <sage@redhat.com>

common/config_proxy: Mutex -> ceph::mutex

Signed-off-by: Sage Weil <sage@redhat.com>