Merge pull request #27395 from Devp00l/wip-issue-38960

nautilus: mgr/dashboard: Fixes tooltip behavior

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #27128 from smithfarm/wip-38865-nautilus

nautilus: ceph-mgr:  ImportError: Interpreter change detected - this module can only be loaded into one interprer per process.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27227 from pdvian/wip-38855-nautilus

nautilus: mgr/balancer: Python 3 compatibility fix

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #27302 from pdvian/wip-38906-nautilus

nautilus: osd/PGLog.h: print olog_can_rollback_to before deciding to rollback

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27319 from sebastian-philipp/wip-39083-nautilus

nautilus: mgr/deepsea: use ceph_volume output in get_inventory()

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27346 from liewegas/wip-cluster-log-to-file-nautilus

nautilus: mon: add cluster log to file option

Merge pull request #27347 from liewegas/wip-ok-to-stop-nautilus

nautilus: mon: ok-to-stop commands for mon and mds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27361 from s0nea/wip-38864-nautilus

nautilus: mgr/dashboard: Fix env vars of `run-tox.sh`

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27363 from s0nea/wip-39002-nautilus

nautilus:  mgr/dashboard: Filter iSCSI target images based on required features

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #27368 from s0nea/wip-39104-nautilus

nautilus: dashboard: NFS: failed to disable NFSv3 in export create

Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #27370 from s0nea/wip-39105-nautilus

nautilus: mgr/dashboard: Cannot submit NFS export form when NFSv4 is not selected

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #27371 from s0nea/wip-39106-nautilus

nautilus: mgr/dashboard: NFS Squash field should be required

Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #27372 from s0nea/wip-39107-nautilus

nautilus: mgr/dashboard: Error creating NFS export without UDP

Reviewed-by:  Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fixes tooltip behavior

The problem was that the tool tip element was added to the current parent
element which caused the CSS to make the last
button in a button group look like the fore last button as a rectangle
but the last element should have a rounded corner.

Fixes: https://tracker.ceph.com/issues/38932
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit 4b23b78)

Merge pull request #27258 from pdvian/wip-38861-nautilus

nautilus: backport krbd discard qa fixes to nautilus

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #27287 from s0nea/wip-39052-nautilus

nautilus: mgr/dashboard add polish language

Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #27382 from tchaikov/nautilus-multi-python3

nautilus: rpm,cmake: use specified python3 version if any

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

rpm: pass %{python3_version} to cmake

to force cmake to use the python3 and python3 modules for building
python3 bindings

on the debian side, it's okay to continue using "-DWITH_PYTHON3=ON", as

- cmake does normalize "ON" to 3
- debian's cmake extension lives on /usr/lib/python3/dist-packages/
  not in a specific /usr/lib/python3.x/dist-packages directory

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7681a38ce51743a19ec5679d49b362f04e271ddb)

cmake: check for libpython of the same version of interpreter

actually cython or python3-cython's dependencies should take care of it.
but we should at least get it right on our side if we check it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c961e002d22ab7ceec20f1019f8c558ec9fcce6c)

cmake: use specified python3 version if any

use might have multiple python3 installed, some of them has/have all
dependencies installed and is good enough for building Ceph. we should
not always use the latest python installed in the system and complain that
there is missing dependencies, even if user has installed all the
python3 dependencies for the older python3.

put in other words, if user only installs cython module for python3.4, but
she has both python3.6 and python3.4 in her system. we should not force
her to uninstall python3.6 for installing Ceph.

this change also aligns with MGR_PYTHON_VERSION. i am not applying the
same change to WITH_PYTHON2, because python2 is already stablized. and distros
are not likely to release new python2 releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d8c13e86048b7aac458f3eda101f9778e3d45dcf)

mgr/dashboard: Fix deletion of NFS transports properties

Fixes: http://tracker.ceph.com/issues/39090
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit a304f5db9f1ccf66dfac5d7ba00bc710660770cf)

mgr/dashboard: Fix NFS squash default value

Fixes: http://tracker.ceph.com/issues/39064
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 0084299273bd3702b55038d02523a7ad3a1f49cc)

mgr/dashboard: Fix NFS pseudo validation

Fixes: http://tracker.ceph.com/issues/39063
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 4139d4b58e37b29f7d84e641ce0c9fd0ee346b8f)

mgr/dashboard: Fix deletion of NFS protocol properties

Fixes: http://tracker.ceph.com/issues/38997
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit f47791e32a2fbd6d2114c33d90f16086ba6b5af9)

mgr/dashboard: iscsi: feature names in error messages

Required/supported RBD features should be included in
the error messages.

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit f570f8a19ddc656c6df911eece77f7ed23720680)

mgr/dashboard: Validate iSCSI images features

Will also filter the list of available images.

Fixes: https://tracker.ceph.com/issues/38074
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit f272429e83aebb1e27cba1db3da6aeaf94506aa5)

mgr/dashboard: Fix env vars of `run-tox.sh`

Fixes: http://tracker.ceph.com/issues/38798
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
(cherry picked from commit 5dee0b6a87753d8b7ff295f9a29c6eb407d17ccd)

Merge pull request #27191 from theanalyst/nautilus-scheduler-cmake

nautilus: rgw: build async scheduler only when beast is built

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27296 from theanalyst/nautilus-sse-c-fixes

nautilus: sse c fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/MonMap: add min_quorum_size() helper

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c143b9a3db4492e5a2ee7919b22581955fc2a119)

mon/MDSMonitor: add 'mds ok-to-stop' command

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aa33a26e32d270286ee110f57cfcf63df63fa913)

mon: add 'mon ok-to-{stop,add-offline,rm}' commands

Helpers to decide when it is safe to stop a mon, add a mon that is
not started, or remove a mon.  (Adding and start a mon would always
be safe, but it takes time to sync, so it's not really possible to do
quickly.)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cfba0acc01230423252f674d11929bf505dc1ec7)

Merge PR #27314 into nautilus

* refs/pull/27314/head:
mon: quiet devname log noise

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27320 into nautilus

* refs/pull/27320/head:
valgrind.supp: add clauses for uninitialised gcm.Xi in OpenSSL.
src/valgrind.supp: replace with the teuthology's whitelist.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mon/LogMonitor: add mon_cluster_log_to_file bool option

Allow cluster logging to a file to be disabled via a boolean.  Default
to true to avoid any change in behavior.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 978ac6e2f8e9fe4cc20410c6b45b1a164e22c4ea)

valgrind.supp: add clauses for uninitialised gcm.Xi in OpenSSL.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 35274eba7a90ffed05561737cb935bb3dc049c2d)

src/valgrind.supp: replace with the teuthology's whitelist.

This change drops src/valgrind.supp file and transplates
the current teuthology's whitelist into qa/valgrind.supp.

This commit has sibiling in the teuthology repo dropping
the file there.

The rationale behind the entire change is duplication of these
files between two repos, their inconsistencies and confusion.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d10ab9ebfdb4aa5ae285f1e2b89fd490584b0cb3)

Merge pull request #27261 from s0nea/wip-38969-nautilus

nautilus: mgr: Configure Py root logger for Mgr modules

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27267 from s0nea/wip-38988-nautilus

nautilus: mgr/dashboard: Add refresh interval to the dashboard landing page

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/deepsea: use ceph_volume output in get_inventory()

DeepSea is being updated to use ceph_volume internally (see
https://github.com/SUSE/DeepSea/pull/1517).  Once this is done,
the mgr_orch.get_inventory runner will just be returning the
raw ceph_volume output, so this commit updates the DeepSea mgr
module to match.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit b0f9c4b5101f92b9e68a6512a9c05bfd345baf2d)

mgr/deepsea: log salt job failures

If a request to salt succeeds, but the event comes back with some internal
failure (e.g.: an exception was thrown inside the salt runner), we need to
log this so the admin can find out what on earth went wrong.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 78c3d65ca9092bdab51ae1e6b6beb3ee37e72938)

mgr/deepsea: use empty strings rather than None for option defaults

This makes the _config_valid() function work properly; the option
defaults are being strinified somewhere, so we need to use empty
strings rather than None (which becomes "None"), in order to check
if they're set or not.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit e81f109bf93317b998d3ea31d64381d9f491a6ef)

mon: quiet devname log noise

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3221381d9fb924e397c3d031c7d4751490409bca)

Merge PR #27167 into nautilus

* refs/pull/27167/head:
mgr/diskprediction_cloud: Correct base64 encode translate table

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27216 into nautilus

* refs/pull/27216/head:
msg/async v2: make v2 work on rdma.

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge PR #27220 into nautilus

* refs/pull/27220/head:
osd/PG: move '}' to the proper place
doc: Document new pg state and changes to auto repair behavior
osd, test: Add num_shards_repaired to osd_stat_t for pushes with repair set 3(3)
osd: Track num_objects_repaired in pg stats 2(3)
test, osd: Improvements to auto_repair 1(3)
test: osd-scrub-repair.sh: use corrupt_and_repair_lrc for lrc tests
osd: Publish stats after all changes made
osd: Fixes for 64-bit PG state

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27240 into nautilus

* refs/pull/27240/head:
mgr/diskprediction_cloud: Correct base64 encode translate table

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27278 into nautilus

* refs/pull/27278/head:
log,global: do not start flusher thread until after we have our mon config
log: buffer log entries until flusher thread starts
log: open log file from flusher thread
common/ceph_context: fix log_to_file observer
common: add bool log_to_file option

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

osd/PGLog.h: print olog_can_rollback_to before deciding to rollback

This is particularly important as the crt might have changed before
being passed to merge_object_divergent_entries().

Fixes: http://tracker.ceph.com/issues/38894
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit aa49c57f3942c676d333e1841886881af125ea01)

Merge pull request #27237 from s0nea/issue_38265_backport

nautilus: mgr/dashboard/qa: Improve tasks.mgr.test_dashboard.TestDashboard.test_standby

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27246 from s0nea/wip-38974-nautilus

nautilus: ERROR: test_disable (tasks.mgr.dashboard.test_mgr_module.MgrModuleTelemetryTest)

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27260 from pdvian/wip-38866-nautilus

nautilus: mgr/dashboard: fix sparkline component

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

rgw: fixup_range() clamps offsets to valid part range

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4b3bac20d550cae274c07a1d555751483dde463b)

test_rgw_crypto: add invalid ranges beyond obj boundary

This is just to assert we're not doing an invalid memory access

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b34a00de0f6ecea6d97fd80f65d2a2dca0876e6e)

tests: rgw crypto tests with partlen vector

4 tests are added when either obj_size for an aligned object, and when an object
is not aligned to a boundary (1 B last part), or part_size is not aligned to a
boundary. Comments on boundary case explains the logic on choosing whether to
align to a 4095B boundary or not

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 1bf59f68db608084a1e7a05aca4cca8a8ab76062)

test_rgw_crypt: BlockCryptNone takes a block_size param

So that block sizes like 4096 can be tested

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9e13a42daaba26751f56f2454f9ec314d7d0d460)

rgw_crypt: make decrypt's parts_len protected

This is so that testing classes can set this and use that to test the branches
in fixup_range tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 78b37a5ccd67c464f3c6e61568e9e827c327ad6d)

rgw_crypt: take care of start & end boundaries when parsing range

As http range header is inclusive of start and end ranges, this otherwise breaks
at ranges close to multipart boundaries

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9c92b56d77932283ccc0336645dc3b76bd0fffa2)

rgw: decrypt filter does not cross multipart boundaries

multipart uploads with sse encrypts each part separately, using an
initialization vector based on the part offset

decryption must respect the same part boundaries, and start each part
with a fresh initialization vector. this means that the decrypt filter
must flush data up to part boundaries before starting the next

Fixes: http://tracker.ceph.com/issues/38700
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b782902954abd20e49c0ed1c14cebee4c8ab6f88)

Merge PR #27253 into nautilus

* refs/pull/27253/head:
qa/tests added mimic-x-singleton suite to rados as symlink

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PG: move '}' to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 0fb951963ff9d03a592bad0d4442049603195e25)

doc: Document new pg state and changes to auto repair behavior

Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 769cdc8294c74b33fe99dd2cdb071b0ebcc9ae69)

osd, test: Add num_shards_repaired to osd_stat_t for pushes with repair set 3(3)

Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 57abdb11fae9f94417e1ee6c9427a33f729a9703)

osd: Track num_objects_repaired in pg stats 2(3)

Leave repair pg state on until recovery finishes or a new scrub starts

Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d2ca3d2feb442f97ca89023c7d01178d96f517a6)

test, osd: Improvements to auto_repair 1(3)

Allow auto_repair for replicated bluestore pools
Regular scrub within auto repair parameters will trigger deep scrub
New state failed_repair if PG repair attempt could not fix everything
Set failed_repair if not possible to repair anything

Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2202e5d0b107795837ce79ffce2a980e8c12fc62)

test: osd-scrub-repair.sh: use corrupt_and_repair_lrc for lrc tests

Fix for argument handling of create_ec_pool()
Always pass a value for allow_overwrites for consistency

Caused by: 3ca750d41dfe33c6efea4abc96d2bd426a9742b9

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 315d32488963705f28cd51a327281b8940d1e7cd)

osd: Publish stats after all changes made

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 06e6ebc7836d0bc65361a7857ba8cf4a31040bc2)

osd: Fixes for 64-bit PG state

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 12bff373d29f4cec8932aa1460da98dca711ec91)

mgr/dashboard: Add polish translation

Add polish translation.
Removed 5 unused trans-units.

Fixes: https://tracker.ceph.com/issues/38236
Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit 53d84ad5d36f99a3d8c49ec1f8b60aa0ed4d248c)

Merge PR #27225 into nautilus

* refs/pull/27225/head:
osd/OSDMap: add log for better debugging
osd/OSDMap: calc_pg_upmaps - restrict optimization to origin pools only
crush: add root_bucket to identify underfull buckets

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27217 into nautilus

* refs/pull/27217/head:
common/config: parse --default-$option as a default value

Reviewed-by: Sébastien Han <seb@redhat.com>

Merge PR #27215 into nautilus

* refs/pull/27215/head:
mon/MonClient: do not dereference auth_supported.end()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27197 into nautilus

* refs/pull/27197/head:
msg: default to debug_ms=0

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27139 into nautilus

* refs/pull/27139/head:
os/bluestore: unconditionally cap chunks returned by allocator to 2^31
os/bluestore: start using 64-bit intervals for bitmap allocator
os/bluestore: make bluestore interval base template.
tests/fastbmap_alloc: UT to reproduce 4G allocation bug
os/bluestore: os/bluestore: implement dump for bitmap allocator
os/bluestore be more tolerant to lack of space for bluefs.

Reviewed-by: Sage Weil <sage@redhat.com>

log,global: do not start flusher thread until after we have our mon config

We want to respect the settings in the mon config that affect logging before
we enable the flusher thread.  That allows us to set (via the monitor) things
like log_to_file=false or log_to_syslog=true.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cd6a5b9c40779956629803f222c365bdb291a169)

log: buffer log entries until flusher thread starts

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4af24e57aafd8774da4b1d2c7e3b17dcae1c18a0)

log: open log file from flusher thread

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 16c5d828fa93b1a5bdf9c396d7c0ce236b46df9b)

common/ceph_context: fix log_to_file observer

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d8810ab58241b80776e2564bce911299341f73e7)

common: add bool log_to_file option

This allows us to disable and reenable logging to a file while preservng
the default log_file location.  This is analogous to log_to_stderr,
log_to_syslog, log_to_graylog, etc.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 18df18aa4d69232f1e898c99522aad7444c9ff12)

qa/tests added mimic-x-singleton suite to rados as symlink

replaces https://github.com/ceph/ceph/pull/27131

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboar: Add refresh  nterval to the dashboard landing page

Fixes: http://tracker.ceph.com/issues/26872
Signed-off-by: guodan1 <guodan1@lenovo.com>
(cherry picked from commit 58e9d48259ad7ac562ffb85bc22b3d6fba6120e1)

mgr: Configure Py root logger for Mgr modules

Add the CPlusPlusHandler to the Python root logger, thus the Mgr module itself and all 3rd party libraries will log to the Ceph log. This may help to identify problems in used 3rd party libraries.

Fixes: https://tracker.ceph.com/issues/38888
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit b0be9475f1d9be9538183e636569ec851801dccb)

mgr/dashboard: fix sparkline component

Fixes: https://tracker.ceph.com/issues/38768
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 36997f93a9fe1b56646bb1f6574ac204467a74ff)

qa: add krbd_discard_granularity.t test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a337cc58cdee1d0443744eef2a43d86738b3124d)

qa: update and rename krbd_discard_1b.t

Passing 1 for alloc_size is no longer allowed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 481b6c2146ac891787ffae6deb2fc77e9cb56542)

Merge pull request #27161 from cbodley/wip-qa-rgw-nautilus

nautilus: qa/rgw: point test repos at ceph-nautilus branch

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

mgr/dashboard/qa: Increase timeout for test_disable (tasks.mgr.dashboard.test_mgr_module.MgrModuleTelemetryTest)

Fixes: https://tracker.ceph.com/issues/38820
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 91fe2f07997d292e42005001e07dbf1dfb94d0c8)

mgr/diskprediction_cloud: Correct base64 encode translate table

Correct base64 encode/decode string translate table

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
(cherry picked from commit 9df1f3cfaaa3fb1de35ed729cc7548da19392255)

Merge pull request #27189 from smithfarm/wip-38903-nautilus

nautilus: doc: Minor rados related documentation fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27219 from LenzGr/wip-38871-nautilus

nautilus: mgr/dashboard: dashboard giving 401 unauthorized

mgr/dashboard/qa: Improve tasks.mgr.test_dashboard.TestDashboard.test_standby

The changes of this PR were done while trying to fix the failing test. The problem has been solved by another PR, but the changes are worth to be integrated because they help debugging and an additional test has been added (check if previously active manager is listed as standby).

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit c29e587007e461738d4f95938dc2eb44f2459964)

osd/OSDMap: add log for better debugging

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a89281ffbb50a4dfc700398e864138b5faaf00f5)

osd/OSDMap: calc_pg_upmaps - restrict optimization to origin pools only

The current implementation will try to cancel any pg_upmaps that
would otherwise re-map a PG out from an underfull osd, which is wrong,
e.g., because it could reliably fire the following assert:

src/osd/OSDMap.cc: 4405: FAILED assert(osd_weight.count(i.first))

Also it would not match the expectation if automatic balancing
has been strictly restricted to some specific pools by admin.

Fix by excluding any wild PG that does not belong to the origin pools
passed in when trying to do upmap/unmap.

Fixes: http://tracker.ceph.com/issues/38897
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 01e8e9482ce7194d347e02ef41acfa6d8d14f614)

crush: add root_bucket to identify underfull buckets

All underfull buckets under root_buckets will be taken as target

For the crule rule:
    step take datacenter0
    step chooseleaf firstn 2 type host
    step emit
    step take datacenter1
    step chooseleaf firstn 2 type host
    step emit

If one host contains overfull osd but no underfull osd,
it will use other underfull buckets as target, which
maybe not in the same datacenter, that will
broke the rule.

Fixes: http://tracker.ceph.com/issues/38826
Signed-off-by: huangjun <huangjun@xsky.com>
(cherry picked from commit 3d5678d3561d90a10d9de3cb6e7e0405dbe8fdfe)

mgr/balancer: Python 3 compatibility fix
Fixes: http://tracker.ceph.com/issues/38831
Signed-off-by: Marius Schiffer <marius@mschiffer.de>
(cherry picked from commit 2065d470e6585c9a69b4a807e9616e1d9ab31589)

common/config: parse --default-$option as a default value

Sometimes it is useful to specify an alternative default value for an
option via the command line such that it has a lower priority than the
mon config database, config file, the rest of the command line, or the
environment.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fdd2000631b25fd964e94373488e7f94694d7020)

msg/async v2: make v2 work on rdma.

When exec " MON=1 OSD=1 RGW=0 MDS=0 MGR=0 ../src/vstart.sh -b -n -o "ms_type=async+rdma"  -o "ms_async_rdma_device_name=mlx5_1"  --msgr2 -o "debug ms=2/2" ", met the following error messages:

2019-03-18 10:04:17.074 7f6644dd8700  1 --2-  >> v2:10.239.48.9:40998/0 conn(0x7f66401691f0 0x7f6640169630 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_client_ident getsockname reveals I am : when talking to v2:10.239.48.9:40998/0
/home/ceph/jp-ceph/src/msg/msg_types.h: In function 'void entity_addr_t::set_port(int)' thread 7f6644dd8700 time 2019-03-18 10:04:17.079278
/home/ceph/jp-ceph/src/msg/msg_types.h: 363: abort()
 ceph version 14.1.1-196-g9c024bafa3 (9c024bafa3570dee401d3fcaf04754376f285eca) nautilus (rc)
 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xfe) [0x7f664cc6e896]
 2: (entity_addr_t::set_port(int)+0xaa) [0x7f664cc3a724]
 3: (ProtocolV2::send_client_ident()+0x3ba) [0x7f664d005270]
 4: (ProtocolV2::finish_client_auth()+0x4c) [0x7f664d004d8e]
 5: (ProtocolV2::handle_auth_signature(ceph::buffer::v14_2_0::list&)+0x612) [0x7f664d00ad70]
 6: (ProtocolV2::handle_frame_payload()+0x275) [0x7f664cffde79]
 7: (ProtocolV2::handle_read_frame_dispatch()+0x158) [0x7f664cffcbea]
 8: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr<ceph::buffer::v14_2_0::ptr_node, ceph::buffer::v14_2_0::ptr_node::disposer>&&, int)+0x80d) [0x7f664cffef77]
 9: (CtRxNode<ProtocolV2>::call(ProtocolV2*) const+0x8c) [0x7f664d02ac40]
 10: (ProtocolV2::run_continuation(Ct<ProtocolV2>&)+0x5e) [0x7f664cff144e]
 11: (()+0x175a34b) [0x7f664cff834b]
 12: (()+0x1774e00) [0x7f664d012e00]
 13: (std::function<void (char*, long)>::operator()(char*, long) const+0x61) [0x7f664cfb6269]
 14: (AsyncConnection::process()+0xc77) [0x7f664cfb16f7]
 15: (C_handle_read::do_request(unsigned long)+0x28) [0x7f664cfb5472]
 16: (EventCenter::process_events(unsigned int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*)+0x678) [0x7f664d02e4cc]
 17: (()+0x179dbf4) [0x7f664d03bbf4]
 18: (()+0x179f0c7) [0x7f664d03d0c7]
 19: (std::function<void ()>::operator()() const+0x32) [0x7f664d03a44a]
 20: (void std::__invoke_impl<void, std::function<void ()>>(std::__invoke_other, std::function<void ()>&&)+0x20) [0x7f664d039ce7]
 21: (std::__invoke_result<std::function<void ()>>::type std::__invoke<std::function<void ()>>(std::function<void ()>&&)+0x26) [0x7f664d039474]
 22: (decltype (__invoke((_S_declval<0ul>)())) std::thread::_Invoker<std::tuple<std::function<void ()> > >::_M_invoke<0ul>(std::_Index_tuple<0ul>)+0x28) [0x7f664d03b3aa]
 23: (std::thread::_Invoker<std::tuple<std::function<void ()> > >::operator()()+0x1d) [0x7f664d03b37b]
 24: (std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::function<void ()> > > >::_M_run()+0x1c) [0x7f664d03b35a]
 25: (()+0xbd57f) [0x7f664af5257f]
 26: (()+0x76db) [0x7f6657d996db]
 27: (clone()+0x3f) [0x7f66580d288f]

This because getsockname can't work on connection->cs.fd(). So add new
api socket_fd() for rdma socket fd.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit de39c8534be644f760d7e7ccee7b782349237c1d)

mon/MonClient: do not dereference auth_supported.end()

if we are unable to find a supported method in allowed_methods, we will
dereference `auth_supported.end()` for searching it in desperation.

Reported-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 998b1925bd6d19b2b32141b92de66b77027130a0)

Merge pull request #27171 from yuriw/wip-yuriw-38819-nautilus_2

qa/tests: initial checkin for nautilus-p2p

qa/tests: initial checkin for nautilus-p2p
symlinked `objectstore` folder

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>