Merge pull request #29165 from dillaman/wip-40865

cls: reduce log level for non-fatal errors

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #28988 from zhangsw/rgw-fix-bug-listobjectv2

rgw: continuation token doesn't work in list object v2 request.

Merge pull request #27420 from pooja-gautam/wip-dashboard-progress-bar

mgr/dashboard Displays progress bar in notification tray for background tasks

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29135 from dillaman/wip-40486

doc/rbd: re-organize top-level and add live-migration docs

Reviewed-by: Mykola Golub <mgolub@suse.com>

cls/journal: reduce verbosity of debug logs for non-errors

Fixes: http://tracker.ceph.com/issues/40865
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: reduce verbosity of debug logs for non-errors

Fixes: http://tracker.ceph.com/issues/40865
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard Displays progress bar in notification tray

Signed-off-by: Pooja <pooja.gautam@ts.fujitsu.com>

doc/rbd: initial live-migration documentation

Fixes: http://tracker.ceph.com/issues/40486
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28764 from rhcs-dashboard/rgw-bucket-placement-target

mgr/dashboard: select placement target on RGW bucket creation

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29100 from tchaikov/wip-cmake-python-osx

cmake: use latest FindPython*.cmake

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #29012 into master

* refs/pull/29012/head:
os/bluestore: proper locking for BlueFS prefetching

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29068 into master

* refs/pull/29068/head:
os/bluestore: cleanup around allocator calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29072 into master

* refs/pull/29072/head:
qa/suites/upgrade: set pg_autoscale_mode=off on existing pools
mon/MgrMonitor: make pg_autoscaler always on
qa/tasks/ceph.conf.template: osd_pool_default_pg_autoscale_mode = off

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29085 into master

* refs/pull/29085/head:
os/bluestore: add slow op detector for collection listing
os/bluestore: parametrize latency threshold for log_latency funcs..
os/bluestore: cleanup around slow op logging.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29145 into master

* refs/pull/29145/head:
doc/release/nautilus: 14.2.2 changes redone

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

doc/dev/macos.rst: update to be in sync with master

* we've moved to cmake 3.5, so no need to hack for newer cmake
* nss dependency is dropped, no need to install it anymore
* pass clang and clang++ using cmake variable to be more consistent
* pass CMAKE_EXE_LINKER_FLAGS as `/usr/local/opt/llvm/lib` is
  not in default library paths.
* mention https://github.com/boostorg/atomic/issues/15

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: pass PYTHON_VERSION explicitly

allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: update FindPython* modules

use the ones shiped from the latest cmake. which

* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
  simpler this way and less error prone.

and to accomodate this change:

* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
  `Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
  `find_package(Python3...)` to find python2 or python3 instead of using
  `find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
  defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
  variables. because we find the python interpreter and development
  files using `find_pacakge(Python...)` for ceph-mgr.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29080 from tchaikov/wip-pybind

pybind: check "CEPH_LIBDIR" not "MAKEFLAGS"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28763 from tchaikov/wip-dpdk-fix-ftbfs

msg/async/dpdk: fix FTBFS

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29120 from neha-ojha/wip-add-write-omap

qa: add basic omap testing capability

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/release/nautilus: 14.2.2 changes redone

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/perf: add op_size to radosbench_omap_write.yaml

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #29123 into master

* refs/pull/29123/head:
test_sessionmap: use sudo_write_file() from teuthology.misc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28946 from cbodley/wip-rgw-list-v1-close-section

rgw: add missing close_section in send_versioned_response

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

Merge pull request #29101 from valentinbajrami/master

mgr/dashboard: Improve e2e script

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Rafael Quintero <rquinter@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

qa/suites/upgrade: set pg_autoscale_mode=off on existing pools

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rbd: tweak config reference

Clean up the caching sections to match the current implementation and
clarify which sections apply only to librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: select placement target on RGW bucket creation

* Select a placement target from the zone that the RGW daemon is running on.

Fixes: https://tracker.ceph.com/issues/40567
Signed-off-by: alfonsomthd <almartin@redhat.com>

doc/man: note that rbd-fuse is not intended for production workloads

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: restructure into operations and integrations

This avoids the long table of contents list of intermingled subjects.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29129 from tchaikov/wip-remove-sepia-bt.sh

script/sepia_bt.sh: remove stale script

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

script/sepia_bt.sh: remove stale script

this script is not actively maintained anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29057 from cyx1231st/wip-seastar-msgr-loggingv2

crimson/net: clean-up and fixes of messenger

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29105 from tspmelo/wip-prevent-propagation

mgr/dashboard: Disable event propagation in the helper icon

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29107 from ricardoasmarques/label-col-size-mgr-module-form

mgr/dashboard: Increase column size on mgr module form

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29112 from ricardoasmarques/iscsi-gateway-status-health-page

mgr/dashboard: Show iSCSI gateways status in the health page

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #28942 from ricardoasmarques/validate-iscsi-controls

mgr/dashboard: Validate iSCSI controls min/max value

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28898 from ricardoasmarques/cephfs-client-evict

mgr/dashboard: Evict a CephFS client

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve e2e script

Passing -r flag to jq to remove quotes. sed not needed
Using $() command substitution instead of the legacy and unsafe backticks `...`

Signed-off-by: Valentin Bajrami <valentin.bajrami@gmail.com>

Merge pull request #28081 from sebastian-philipp/kubejacker-doc-update

doc/dev/kubernetes: Update

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #29125 from bk201/wip-40827

mgr/dashboard: fix tox test failure

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Evict a CephFS client

Fixes: https://tracker.ceph.com/issues/24892
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Show iSCSI gateways status in the health page

Fixes: https://tracker.ceph.com/issues/39028
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: fix tox test failure

`api/task` now also returns tasks in progress module.
Mock related function for tests.

Fixes: https://tracker.ceph.com/issues/40827
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

test_sessionmap: use sudo_write_file() from teuthology.misc

Instead of defining a method to write files within the testsuite, use
sudo_write_file() from teuthology.misc.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #29067 into master

* refs/pull/29067/head:
qa/tasks: add exception in do_thrash()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks: add exception in do_thrash()

Fixes the bug during DaemonWatchdog testing:

Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-daemonwatchdog-testing13/qa/tasks/daemonwatchdog.py", line 38, in _run
    self.watch()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-daemonwatchdog-testing13/qa/tasks/daemonwatchdog.py", line 112, in watch
    if thrasher.e is not None:
AttributeError: MonitorThrasher instance has no attribute 'e'

Signed-off-by: Jos Collin <jcollin@redhat.com>

qa/suites/rados/perf: add objectsize to radosbench_omap_write.yaml

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge PR #29011 into master

* refs/pull/29011/head:
doc/releases/nautilus: move note to 14.2.2
doc/releases/nautilus: typo
doc/releases/nautilus.rst: pool stats fix was merged for 14.2.2 after all
doc: release notes for nautilus 14.2.2

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #28692 into master

* refs/pull/28692/head:
test_volume_client: add positive test for ceph_volume_client method
test_volume_client: rename test_put_object_versioned()
test_volume_client: rewrite test_put_object_versioned
test_volume_client: use sudo_write_file() form teuthology
test_volume_client: make test_object_versioned py3 compatible

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/rados/perf: add radosbench_omap_write.yaml

This excercises the --write-omap option in the radosbench task.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22788 from adamyanova/wip-s3java

A task to run S3 Java tests against RGW

qa: add basic omap testing capability

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29048 from rjfd/wip-dashboard-progress-tasks

mgr/dashboard: integrate progress mgr module events into dashboard tasks list

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #28006 from Devp00l/wip-39482

mgr/dashboard: Watch for pool pg's increase and decrease

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #29092 into master

* refs/pull/29092/head:
osd/ECTransaction,ReplicatedBackend: create op is new in octopus

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28935 from votdev/issue_40248

mgr/dashboard: Allow users to change their password on the UI

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #28928 from p-na/wip-pna-e2e-pools

mgr/dashboard: Write E2E tests for pool creation, deletion and verification

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Rafael Quintero <rquinter@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Increase col size on mgr module edit form

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Disable event propagation in the helper icon

Fixes: http://tracker.ceph.com/issues/40715
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28882 from rjfd/wip-mgr-perf-counter-timestamp

mgr: return perf_counters data timestamps in nanosecs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc/releases/nautilus: move note to 14.2.2

Oops! This change was merged in 404ed645d42471b675f11fb7d11f46c006248152,
which was not part of 14.2.1 after all.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases/nautilus: typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29069 into master

* refs/pull/29069/head:
qa/tasks/ceph_manager: make upmap thrasher behave when no pools/pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29095 into master

* refs/pull/29095/head:
script/ptl-tool: update for python3

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #28900 into master

* refs/pull/28900/head:
doc: Update troubleshooting.rst
doc: drop kernel-client-troubleshooting.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: update for python3

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29071 into master

* refs/pull/29071/head:
selinux: Allow ceph to read udev db

Reviewed-by: Sage Weil <sage@redhat.com>

osd/ECTransaction,ReplicatedBackend: create op is new in octopus

Not nautilus!

Fixes: https://tracker.ceph.com/issues/40755
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #28987 into master

* refs/pull/28987/head:
Revert "mgr/dashboard: Add nolockdep option to e2e-script"
common: refactor handling of lockdep's group name in debug locks.
common: shared_mutex_debug doesn't use empty group name for lockdep.
common, crypto: give names to OpenSSL's mutexes.
common: tiny_vector – CPU-friendly container for mutexes & co.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29051 into master

* refs/pull/29051/head:
mon/OSDMonitor: tolerate duplicate MRemoveSnaps messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28468 from jmolmo/ansible_orchestrator

mgr/ansible: RGW service

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr: return perf_counters data timestamps in nanosecs

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend: move ellipsis to before progress in task execution description

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: test_pool: fix pool unit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add rudimentary pool create/exist/delete test

Fixes: http://tracker.ceph.com/issues/38093
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge branch 'master' into ansible_orchestrator

os/bluestore: add slow op detector for collection listing

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: parametrize latency threshold for log_latency funcs..

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: cleanup around slow op logging.

We don't need macros any more.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

crimson/net: apply strict checks when handle addresses

* Apply strict checks at each step of addresses exchange, make sure we
  fail early when notice inconsistent addresses sent from peer;
* Throw error::bad_peer_address for address validation from peer;
* Stop learning my address before sending it out for peer as identity;
* Fix potential racing when try to learn my address by submit to core 0;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

pybind: check "CEPH_LIBDIR" not "MAKEFLAGS"

if the used cmake generator is not "make", these environmental variables
won't be set, so let's just check for "CEPH_LIBDIR". it is set by cmake
function of `distutils_add_cython_module()` and
`distutils_install_cython_module()`

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: tests: fix tests by mocking progress service

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: frontend/task.service: format task title for progress tasks

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: add progress module tasks to list of dashboard tasks

Fixes: https://tracker.ceph.com/issues/38202
Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: progress mgr module service

This service module implements helper methods to retrieve
tasks information from the progress mgr module.

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/progress: persist started_at and finished_at timestamps

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/progress: export more event properties to json

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Add SSO guard service.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Validate iSCSI controls min/max value

Fixes: https://tracker.ceph.com/issues/38018
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Allow users to change their password on the UI

Fixes: https://tracker.ceph.com/issues/40248
Signed-off-by: Volker Theile <vtheile@suse.com>

crimson/net: fix crimson-osd msgr policies

Also add checks to make sure we have a valid target address with
lossless policy.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: use target_addr to store the address to reconnect

With lossless policy, we will use target_addr to reconnect to peer,
which would be more useful when we switch to entity_addrvec_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: remove duplicated logging when connected

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: set connection policy for v1 protocol

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/net: use set_ephemeral_port() for non-bound port

Ephemeral port is different from the bound port when connecting, and
different from the peer_addr port when accepting.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #28959 from tchaikov/wip-crimson-op-write

crimson/osd: differentiate write from writefull

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/releases/nautilus.rst: pool stats fix was merged for 14.2.2 after all

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #28861 from smanjara/wip-multi-meta

rgw/multisite: Don't allow certain radosgw-admin commands to run on non-master zone