mgr/dashboard: Escape regex pattern in DeletionModalComponent

Fixes: http://tracker.ceph.com/issues/24902
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23261 from dillaman/wip-vstart

vstart: do not attempt to re-initialize dashboard for existing cluster

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23384 from tchaikov/wip-cmake-cleanup

cmake: fix std::filesystem detection and extract sanitizer detection into its own module

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #23381 into master

* refs/pull/23381/head:
mds: update MDSRank::cluster_degraded before handling mds failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23398 from alfredodeza/wip-patch-release

ceph-volume tests patch __release__ to mimic always for stdin keys

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests patch __release__ to mimic always for stdin keys

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23382 from tchaikov/wip-gtest-parallel-py3

cmake,run-make-check: always enable WITH_GTEST_PARALLEL

Reviewed-by: Nathan Cutler <ncutler@suse.com>

seastar: pick up changes in seastar

-ldl is not necessary if an application using dlopen() can compile and
link just fine without it.

same applies to -lrt, when it comes to time related calls.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23386 from smithfarm/wip-fix-seastar-builddeps

build/ops: rpm: fix seastar build dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23375 from alfredodeza/wip-rm25216

ceph-volume lvm.activate conditional mon-config on prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23328 from votdev/bug_24570

mgr/dashboard: Close modal dialogs on login screen

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

build/ops: rpm: fix seastar build dependencies

1. cryptopp-devel was moved to the distro-specific section by
aeb974b9139d49f13e162cd90da8a0a4b3fc3b33, then
96196e9d774ab55fabb792935a09a81cf0ca9786 reintroduced it in the
non-distro-specific section, breaking install-deps.sh for SUSE

2. fmt-devel is called libfmt-devel on SUSE

Fixes: install-deps.sh on SUSE
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23363 from votdev/fix_typo

mgr/dashboard: Fix typo

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

run-make-check: drop CMAKE_PYTHON_OPTS variable

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake,run-make-check: always enable WITH_GTEST_PARALLEL

now that https://github.com/google/gtest-parallel/pull/63 has been
merged, we can now use gtest-parallel with py2 and also py3.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: extract sanitizer detection out

also, we don't need to pass '-lasan' or '-ltsan' to linker. it's
suggested to use the "-fsanitize=${sanitizer}".

please note, this module is compatible with the one used in seastar on
purpose, as seastar is included in ceph using add_subdirectory(), and it
in turn add its own cmake modules directory using list(APPEND ...), so
cmake/modules/FindSantitizers.cmake is prefered over
src/seastar/cmake/FindSantitizers.cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: fix StdFilesystem detection

before this change, we fails to detect std::filesystem with clang++,
because  the cmake project created by try_compile() only expands
following options passed from its caller:
- COMPILE_DEFINITIONS
- INCLUDE_DIRECTORIES
- LINK_DIRECTORIES
- LINK_LIBRARIES

which do not include CMAKE_CXX_FLAGS, so either we need to (ab)use
COMPILE_DEFINITIONS for passing -std=c++17, or we can change the
CMAKE_CXX_FLAGS in the parent env, as it turns out the created cmake
project does inherit this flag from current project. in this change,
we use the COMPILE_DEFINITIONS approach: simpler this way. and we
can drop it once cmake 3.8 is required.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #21948 into master

* refs/pull/21948/head:
ceph_volume_client: use integer division for pg_num
packages.yaml: install py3 compatible cephfs and rados bindings...
qa: test py2 and py3 for volume client
qa: make test_volume_client.py py3 compatible
qa/ceph-volume: allow executing python pyloads using python3
cephfs.pyx: add py3 compatibility
ceph-volume-client: add py3 compatibility

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22450 into master

* refs/pull/22450/head:
client: LAZY_IO support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23290 into master

* refs/pull/23290/head:
tools/ceph-dencoder: include types.h with full path
tools/ceph-dencoder: move ceph-dencoder to tools/ceph-dencoder

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #23365 into master

* refs/pull/23365/head:
mds/MDCache: fix mds_cache_memory_limit get_val<>

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23380 from majianpeng/cmake-jemalloc-error

cmake: fix a cmake error when with -DALLOCATOR=jemalloc.

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: fix a cmake error when with -DALLOCATOR=jemalloc.

When exec: ./do_cmake.sh -DALLOCATOR=jemalloc. Met the following
messages:
>> CMake Error at cmake/modules/FindJeMalloc.cmake:28 (endforeach):
  endforeach An ENDFOREACH command was found outside of a proper FOREACH
  ENDFOREACH structure.  Or its arguments did not match the opening FOREACH
  command.
>> Call Stack (most recent call first):
  CMakeLists.txt:326 (find_package)

This bug introduce commit 8db629a14baf2aea99cf.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #23300 from tchaikov/wip-cmake-cleanup

cmake: cleanups

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mds: update MDSRank::cluster_degraded before handling mds failure

Migrator checks MDSRank::is_cluster_degraded() to decide if it needs to
send message to other mds. MDSRank::cluster_degraded should get updated
before calling Migrator::handle_mds_failure_or_stop().

Introduced by commit 7de9da4a "mds: handle discontinuous mdsmap"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

ceph-volume lvm.activate conditional mon-config on prime-osd-dir

The --no-mon-config should only be done in Mimic releaes and later

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23372 from cbodley/wip-25214

qa/rgw: override valgrind --max-threads for radosgw

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

qa/rgw: override valgrind --max-threads for radosgw

radosgw now uses 512 frontend threads by default, and valgrind won't
start with its default --max-threads=500

Fixes: http://tracker.ceph.com/issues/25214
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23145 from cbodley/wip-qa-rgw-admin-rest

qa/rgw: add radosgw-admin-rest task to singleton suite

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge PR #23195 into master

* refs/pull/23195/head:
mds: use vector for context lists
common: make C_ContextsBase container agnostic
common: make finish_contexts container agnostic
mds: add vector def for MDS contexts
mds: use compact map to manage waiting list

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #23353 into master

* refs/pull/23353/head:
doc: explain how to mkdir under .snap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: explain how to mkdir under .snap

Prior to this change, the CephFS snapshot instructions were not clear
whether users should "mkdir" the ".snap" directory or not.

Update the documentation to clarify that users should make a directory
under .snap with a name of their choice.

Thanks to John Spray <jspray@redhat.com> on ceph-users for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

mds/MDCache: fix mds_cache_memory_limit get_val<>

Fixes: 922bfc5f3b333f576a1444773fff6db7100dd24a
Fixes: http://tracker.ceph.com/issues/25215
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23355 from alfredodeza/wip-rm25173

ceph-volume do not use stdin in luminous

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23366 from tchaikov/wip-cmake-no-common-crc-aarch64

cmake: do not link against common_crc_aarch64

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake: do not link against common_crc_aarch64

it's included by libcrc32 already

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #23340 into master

* refs/pull/23340/head:
osd/PGLog.cc: use lgeneric_subdout instead of generic_dout

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: Fix typo

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #23342 from ZVampirEM77/wip-sync-error-trim-param

rgw-admin: add "--trim-delay-ms" introduction for 'sync error trim'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23360 from badone/wip-release-schedule-update-august-2018

doc/releases: Update releases to August '18

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

ceph-volume tests verify that luminous doesn't use stdin

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util do not use stdin for luminous

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23224 from votdev/rest_client_timeout

mgr/dashboard: Set timeout in RestClient calls

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #21881 from sebastian-philipp/dashboard-pool-patch

mgr/dashboard: Add Pool update endpoint

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Modal dialogs are still open when UI is redirected to the login screen

Fixes https://tracker.ceph.com/issues/24570

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #23284 from tchaikov/wip-seastar-config

crimson/common: write configs synchronously on shard.0

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/releases: Update releases to August '18

Mimic    13.2.1
Luminous 12.2.5, 12.2.6, 12.2.7
Jewel    10.2.11

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

rgw-admin: add "--trim-delay-ms" introduction for 'sync error trim'

Signed-off-by: Enming.Zhang <enming.zhang@umcloud.com>

Merge PR #22825 into master

* refs/pull/22825/head:
common: FreeBSD does not have /etc/os-release

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22998 into master

* refs/pull/22998/head:
filestore: add pgid in filestore pg dir split log message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #23134 into master

* refs/pull/23134/head:
common: check completion condition before waiting

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #23223 into master

* refs/pull/23223/head:
osd/PG: kill dead functions and related options
iosd/osd_type: kill unused input ec_pool for iterate_mayberw_back_to
common: kill dead options
osd/PG: do not initialize up/acting twice
osd/PG: clear missing_loc properly if last location is gone

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22990 from tchaikov/wip-cmake-link-static-libstdc++

cmake: fix "WITH_STATIC_LIBSTDCXX"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #22692 into master

* refs/pull/22692/head:
doc/mgr/devicehealth: document devicehealth module
doc/rados/operations/health-checks: document DEVICE_HEALTH* messages
mgr/devicehealth: fix style for returns
mgr/devicehealth: use constants for health warnings
mgr/devicehealth: deal with as many daemons as we can until limit
mgr/devicehealth: warn if too many daemons are expected to fail soon
mgr/devicehealth: set primary-affinity 0 for failing devices
msg/devicehealth: fix config options
mgr/devicehealth: only fetch osdmap once from check_health
mgr/devicehealth: revise health messages
mgr/devicehealth: add 'device check-health' command and run periodically
mgr/devicehealth: fix new options
mgr/devicehealth: add helpers to life_expectancy_response()
mgr/devicehealth: simplify setting defaults
common/blkdev remove debug statements

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #23157 into master

* refs/pull/23157/head:
Provided API to change umask

Merge PR #23354 into master

* refs/pull/23354/head:
src/osd/PG.cc: remove redundant call to trim_log()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc/mgr/devicehealth: document devicehealth module

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/health-checks: document DEVICE_HEALTH* messages

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: fix style for returns

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: use constants for health warnings

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: deal with as many daemons as we can until limit

Process as many OSDs as we can until we hit the min_in_ratio.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: warn if too many daemons are expected to fail soon

Refuse to mark out *all* OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: set primary-affinity 0 for failing devices

Signed-off-by: Sage Weil <sage@redhat.com>

msg/devicehealth: fix config options

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: only fetch osdmap once from check_health

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: revise health messages

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: add 'device check-health' command and run periodically

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: fix new options

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: add helpers to life_expectancy_response()

- if mark_out_threshold is met we write to log.warn instead of raising a
  health warning.
- check that OSD is 'in' before calling mark_out().
- raise a health warning in case OSD is marked 'out' but still has PGs
  attached to it.
- cast thresholds default values to string.
- add SCSI multipath support to health warning message.
- change health warning message.

Signed-off-by: Yaarit Hatuka <yaarithatuka@gmail.com>

mgr/devicehealth: simplify setting defaults

Signed-off-by: Sage Weil <sage@redhat.com>

common/blkdev remove debug statements

Signed-off-by: Yaarit Hatuka yaarithatuka@gmail.com

Merge PR #23334 into master

* refs/pull/23334/head:
pybind/rados/rados: do not pass prval from stack

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23217 from dzafman/wip-25085

osd: Allow repair of an object with a bad data_digest in object_info on all replicas

Reviewed-by: Sage Weil <sage@redhat.com>

src/osd/PG.cc: remove redundant call to trim_log()

This change is motived by the failure tracked in
https://tracker.ceph.com/issues/25198. The failure highlights a case, when a
call to trim_log() after the PG has recovered, races with the previous op,
on a replica OSD. Since the previous operation has not completed, the
last_complete value for that OSD is not valid, when we try to trim the
log. It is also worth noting that the race is due to MOSDPGTrim going through
the strict queue as a peering message vs regular ops going through the
non-strict queue.

During the investigation of this bug, we noticed that, with
https://tracker.ceph.com/issues/23979, we allow pg log trimming to
happen on the primary and replicas, whenever we cross the upper bound of
the pg log. This also ensures that pg log trimming happens while processing
any new op.

Therefore, the function trim_log(), which earlier served the purpose of
trimming logs on the primary and replicas, just before the PG went into
the Recovered state, is no more required. This acted like a last line of
defense to trim logs, when we did not need the logs any more. But, this call
seems redundant now, because, we are limiting the pg log length at all times.

Signed-off-by: Neha Ojha <nojha@redhat.com>

pybind/rados/rados: do not pass prval from stack

The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23348 from ceph/wip-rm24957

ceph-volume: adds test for `ceph-volume lvm list /dev/sda`

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: adds a unit test for `lvm list /dev/sda`

This test is to prove that the issue from
http://tracker.ceph.com/issues/24957 was fixed
by http://tracker.ceph.com/issues/24784

When running lvm list against a raw device it should handle
gracefully the situation where there are multiple PVs with the
name of the given device.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: move pvolumes fixture into conftest.py

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23336 from noahdesu/vstart-dashboard-no-rbd

vstart: disable dashboard when rbd not built

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PGLog.cc: use lgeneric_subdout instead of generic_dout

Signed-off-by: Neha Ojha <nojha@redhat.com>

ceph_volume_client: use integer division for pg_num

Otherwise a float is sent to the manager which is not the invalid format.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

packages.yaml: install py3 compatible cephfs and rados bindings...

on Ubuntu and CentOS for upstream testing.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: test py2 and py3 for volume client

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: make test_volume_client.py py3 compatible

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa/ceph-volume: allow executing python pyloads using python3

Signed-off-by: Rishabh Dave <ridave@redhat.com>

cephfs.pyx: add py3 compatibility

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume-client: add py3 compatibility

Fixes: http://tracker.ceph.com/issues/17230
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #23297 into master

* refs/pull/23297/head:
ceph_volume_client: add delay for MDSMap to be distributed

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #23308 into master

* refs/pull/23308/head:
doc: s/Ceph FS/CephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

vstart: disable dashboard when rbd not built

dashboard doesn't load correctly without the rbd module, which means
vstart commands that interact with dashboard fail and vstart exits.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>

Merge pull request #23292 from yuriw/wip-yuriw-25140-master

qa/tests: added 1st draft of mimic-x suite

Merge pull request #23302 from yuriw/wip-yuriw-crontab-master

qa/tests: added mimic-x to the schedule

Merge pull request #23321 from cernceph/dvanders_enable

ceph-volume: enable the ceph-osd during lvm activation

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23332 from alfredodeza/wip-rm25171

ceph-volume add a __release__ string, to help version-conditional calls

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23305 from smithfarm/wip-cleanup-upgrade

qa/upgrade: cleanup for nautilus

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests: added 1st draft of mimic-x suite

Fixes: https://tracker.ceph.com/issues/25140
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume add a __release__ string, to help version-conditional calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23276 from tchaikov/wip-config-diff-lock

common/config: fix the lock in ConfigProxy::diff()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23251 from neha-ojha/wip-25112

osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23249 from liewegas/wip-mon-cx-nautilus

osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>