]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoqa/tests: clients' upgrades for older releases must use "-t py2" 36801/head
Yuri Weinstein [Tue, 25 Aug 2020 20:59:42 +0000 (13:59 -0700)]
qa/tests: clients' upgrades for older releases must use "-t py2"

Fixes: https://tracker.ceph.com/issues/47120
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoqa/tests: use "-k distro" for all suites (except krbd) 36569/head
Yuri Weinstein [Tue, 11 Aug 2020 17:53:00 +0000 (10:53 -0700)]
qa/tests: use "-k distro" for all suites (except krbd)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoqa/tests: changes related to #36435 and #36436 36437/head
Yuri Weinstein [Mon, 3 Aug 2020 17:20:28 +0000 (10:20 -0700)]
qa/tests: changes related to #36435 and #36436

Merge only after #36435 and #36436 merged

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoMerge pull request #36429 from tchaikov/wip-ceph-debug-docker-crimson
Kefu Chai [Mon, 3 Aug 2020 15:48:52 +0000 (23:48 +0800)]
Merge pull request #36429 from tchaikov/wip-ceph-debug-docker-crimson

ceph-debug-docker: install ceph-crimson-osd for crimson flavor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #35980 from tspmelo/wip-table-status
Lenz Grimmer [Mon, 3 Aug 2020 15:22:03 +0000 (17:22 +0200)]
Merge pull request #35980 from tspmelo/wip-table-status

mgr/dashboard: Display table status in refresh button

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36401 from tchaikov/wip-pg-stat-ack
Kefu Chai [Mon, 3 Aug 2020 14:47:40 +0000 (22:47 +0800)]
Merge pull request #36401 from tchaikov/wip-pg-stat-ack

messages/MPGStats: print seq instead of tid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36375 from ricardoasmarques/force-registry-auth-config-set
Sebastian Wagner [Mon, 3 Aug 2020 14:26:24 +0000 (16:26 +0200)]
Merge pull request #36375 from ricardoasmarques/force-registry-auth-config-set

cephadm: Fix error bootstraping with '--registry-json'

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36386 from bk201/wip-46045
Sebastian Wagner [Mon, 3 Aug 2020 14:25:44 +0000 (16:25 +0200)]
Merge pull request #36386 from bk201/wip-46045

mgr/cephadm: skip Dashboard configuration if Dashboard is disabled

Reviewed-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36400 from tchaikov/wip-rip-xio-msgr
Kefu Chai [Mon, 3 Aug 2020 14:07:45 +0000 (22:07 +0800)]
Merge pull request #36400 from tchaikov/wip-rip-xio-msgr

cmake: do not check OFED_PREFIX anymore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoceph-debug-docker: install ceph-crimson-osd for crimson flavor 36429/head
Kefu Chai [Mon, 3 Aug 2020 13:18:38 +0000 (21:18 +0800)]
ceph-debug-docker: install ceph-crimson-osd for crimson flavor

would be handy if we can have crimson packages when debugging crimson
flavored branch.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #35060 from yison/rwl-image-writeback-cache-hook-seq1
Jason Dillaman [Mon, 3 Aug 2020 10:42:54 +0000 (06:42 -0400)]
Merge pull request #35060 from yison/rwl-image-writeback-cache-hook-seq1

rbd/cache: Replicated Write Log Hook part 11

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #35659 from agayev/track-per-zone-metadata
Kefu Chai [Mon, 3 Aug 2020 10:12:29 +0000 (18:12 +0800)]
Merge pull request #35659 from agayev/track-per-zone-metadata

os/bluestore: track per zone metadata

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #30838 from ifed01/wip-ifed-single-alloc
Kefu Chai [Mon, 3 Aug 2020 10:00:16 +0000 (18:00 +0800)]
Merge pull request #30838 from ifed01/wip-ifed-single-alloc

os/bluestore: use single allocator for shared bluestore/bluefs device

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #36356 from mgfritch/orch-format-not-supported
Kefu Chai [Mon, 3 Aug 2020 09:58:54 +0000 (17:58 +0800)]
Merge pull request #36356 from mgfritch/orch-format-not-supported

mgr/orch: disallow unimplemented format types

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agolibrbd: enable image cache after getting exclusive lock 35060/head
Yuan Lu [Fri, 19 Jun 2020 10:27:34 +0000 (18:27 +0800)]
librbd: enable image cache after getting exclusive lock

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
4 years agolibrbd: add InitRequest and ShutdownRequest state machine
Yuan Lu [Thu, 11 Jun 2020 11:11:54 +0000 (19:11 +0800)]
librbd: add InitRequest and ShutdownRequest state machine

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
4 years agolibrbd: split ReadRequest
Yuan Lu [Tue, 12 May 2020 09:23:13 +0000 (17:23 +0800)]
librbd: split ReadRequest

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
4 years agolibrbd: add IMAGE_CACHE feature
Yuan Lu [Tue, 9 Jun 2020 16:36:11 +0000 (00:36 +0800)]
librbd: add IMAGE_CACHE feature

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
4 years agoMerge pull request #36417 from tchaikov/wip-crimson-injectargs
Kefu Chai [Mon, 3 Aug 2020 09:26:30 +0000 (17:26 +0800)]
Merge pull request #36417 from tchaikov/wip-crimson-injectargs

crimson/admin: implement "injectargs" command

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agocrimson/osd: ignore SIGHUP 36417/head
Kefu Chai [Mon, 3 Aug 2020 03:04:38 +0000 (11:04 +0800)]
crimson/osd: ignore SIGHUP

ignore SIGHUP, as crimson does not write log messages using the
facilities offered by ceph::logging. it uses seastar::logger instead.

and seastar::logger writes to stderr, stdout and/or syslog. there is no
need to reopen the log file when the settings change.

and we do send SIGHUP to OSD processes when performing thrash tests. to
avoid killing crimson-osd processes by accident. SIGHUP is ignored.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36407 from tchaikov/wip-mgr-autopep8
Kefu Chai [Mon, 3 Aug 2020 01:33:21 +0000 (09:33 +0800)]
Merge pull request #36407 from tchaikov/wip-mgr-autopep8

mgr/cephadm: track dependency in tox.ini

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36378 from orozery/librados_test_stub_assert_version
Jason Dillaman [Sun, 2 Aug 2020 20:58:07 +0000 (16:58 -0400)]
Merge pull request #36378 from orozery/librados_test_stub_assert_version

test/librados_test_stub: handle assert_version

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agocrimson/admin: implement "injectargs" command
Kefu Chai [Sun, 2 Aug 2020 16:53:25 +0000 (00:53 +0800)]
crimson/admin: implement "injectargs" command

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common: add ConfigProxy::inject_args()
Kefu Chai [Sun, 2 Aug 2020 16:52:17 +0000 (00:52 +0800)]
crimson/common: add ConfigProxy::inject_args()

prepare for the "injectargs" tell command.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36406 from tchaikov/wip-pg-stat-seq
Kefu Chai [Sun, 2 Aug 2020 02:19:47 +0000 (10:19 +0800)]
Merge pull request #36406 from tchaikov/wip-pg-stat-seq

crimson/osd: send pg_stat with correct seq

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agomgr/cephadm: track dependency in tox.ini 36407/head
Kefu Chai [Sun, 2 Aug 2020 01:31:01 +0000 (09:31 +0800)]
mgr/cephadm: track dependency in tox.ini

* track dependency for running tox tests with tox.ini. the fix
  test is not performed by "make check", so `install-deps.sh`
  is not responsible for preparing the autopep8 wheels. hence
  we can just put it in tox.ini
* remove "python_version >= '3'" as we do not support python2

this change addresses a regression introduced by
7d8a3b60321da10af6f4557a9d69eda09ca2c620, which fails the rpmbuild like:

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/share/ceph/mgr/requirements-fix.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36348 from votdev/cephadm_tox_pylint
Sebastian Wagner [Sat, 1 Aug 2020 20:36:16 +0000 (22:36 +0200)]
Merge pull request #36348 from votdev/cephadm_tox_pylint

mgr/cephadm: Introduce tox and autopep8

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agocrimson/osd: send pg_stat with correct seq 36406/head
Kefu Chai [Sat, 1 Aug 2020 16:28:36 +0000 (00:28 +0800)]
crimson/osd: send pg_stat with correct seq

* set the osd_stat in pg_stat message sent to mgr, otherwise the seq id
  in the message would be always 0.
* return the seq id composed of up_epoch and osd_stat_seq when serving
  "flush_pg_stats" tell command.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: remove unused #include
Kefu Chai [Thu, 30 Jul 2020 07:07:32 +0000 (15:07 +0800)]
crimson/osd: remove unused #include

these messages are handled by osd/PeeringState

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomessages/MPGStats: print seq instead of tid 36401/head
Kefu Chai [Sat, 1 Aug 2020 10:07:35 +0000 (18:07 +0800)]
messages/MPGStats: print seq instead of tid

osd_stat.seq is a monotonic increasing sequence number which marks
the version of the of the report. since we don't use the tid to track
a MPGStats message and its MPGStatsAck message, it's more interesting to
us to print the sequence id. we use it to verify if a pg_stats is
flushed or not from a certain OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd/OSD: do not include MPGStatsAck.h
Kefu Chai [Sat, 1 Aug 2020 10:04:01 +0000 (18:04 +0800)]
osd/OSD: do not include MPGStatsAck.h

pg_stat report and ack machinery was removed in
528736bed3a6adffa37d0bad6a0e1bedb55db931, and we don't generate or handle
MPGStatsAck anymore since mimic. so remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoos/bluestore: proper BlueFS shared allocator setup 30838/head
Igor Fedotov [Sat, 1 Aug 2020 08:43:44 +0000 (11:43 +0300)]
os/bluestore: proper BlueFS shared allocator setup

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #36395 from dillaman/wip-46679
Mykola Golub [Sat, 1 Aug 2020 07:16:59 +0000 (10:16 +0300)]
Merge pull request #36395 from dillaman/wip-46679

test/librbd: unquiesce notifications are handled lazily

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #36309 from dillaman/wip-46694
Mykola Golub [Sat, 1 Aug 2020 07:15:56 +0000 (10:15 +0300)]
Merge pull request #36309 from dillaman/wip-46694

librbd:  global and pool-level config overrides require image refresh to apply

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agocmake: use add_compile_options for passing compile options 36400/head
Kefu Chai [Sat, 1 Aug 2020 05:33:23 +0000 (13:33 +0800)]
cmake: use add_compile_options for passing compile options

add_definitions() is for, well, adding definitions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agokv: always enable rocksdb
Kefu Chai [Sat, 1 Aug 2020 05:29:52 +0000 (13:29 +0800)]
kv: always enable rocksdb

RocksDB is included as a submodule of Ceph, there is no need to check
for its existence anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not check OFED_PREFIX anymore
Kefu Chai [Sat, 1 Aug 2020 05:17:44 +0000 (13:17 +0800)]
cmake: do not check OFED_PREFIX anymore

OFED_PREFIX was added to compile with xio messenger for supporting RDMA
using the Accelio and/or MLNX_OFED package. but xio messenger was removed in
in cc9a9142fd41ee5c98792c6f6d2e5504

let remove the leftover in CMakeLists

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #32410 from zdover23/wip-doc-pg-repair
zdover23 [Sat, 1 Aug 2020 02:07:12 +0000 (12:07 +1000)]
Merge pull request #32410 from zdover23/wip-doc-pg-repair

doc/rados/operations: document pg repairing practices

4 years agoMerge PR #36364 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:19:46 +0000 (14:19 -0700)]
Merge PR #36364 into master

* refs/pull/36364/head:
client: clean up the unnecessary client_lock for _conf->client_trace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #36359 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:18:52 +0000 (14:18 -0700)]
Merge PR #36359 into master

* refs/pull/36359/head:
mds: place MDSGatherBuilder on the stack

Reviewed-by: Zheng Yan <zyan@redhat.com>
4 years agoMerge PR #36358 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:18:09 +0000 (14:18 -0700)]
Merge PR #36358 into master

* refs/pull/36358/head:
mds: resolve SIGSEGV in waiting for uncommitted fragments

Reviewed-by: Zheng Yan <zyan@redhat.com>
4 years agoMerge PR #36257 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:16:59 +0000 (14:16 -0700)]
Merge PR #36257 into master

* refs/pull/36257/head:
client: move client_lock to _unmount()
client: add timer_lock support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #36155 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:15:22 +0000 (14:15 -0700)]
Merge PR #36155 into master

* refs/pull/36155/head:
qa: Fix traceback during fs cleanup between tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoMerge PR #35944 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:12:53 +0000 (14:12 -0700)]
Merge PR #35944 into master

* refs/pull/35944/head:
qa: defer cleaning the mountpoint's netnses and the bridge
qa/tasks/cephfs/mount.py: remove the stale netnses and bridge
qa/tasks/cephfs/mount.py: try to flush the stale ceph-brx dev info
qa/tasks/cephfs/mount.py: switch to run_shell_payload() helper
qa/tasks/cephfs/mount.py: clean up the none used code

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoMerge PR #35647 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:11:27 +0000 (14:11 -0700)]
Merge PR #35647 into master

* refs/pull/35647/head:
mgr/volumes: Add documentation regarding --retain-snapshots option
mgr/volumes: Avoid trashing retained subvolume on create errors
mgr/volumes: Add subvolume v2 test cases
mgr/volumes: Derive v2 from v1 to leverage common methods
mgr/volumes: Introduce v2 subvolumes
mgr/volumes: Use operation type during subvolume open

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
4 years agoMerge PR #35096 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:10:08 +0000 (14:10 -0700)]
Merge PR #35096 into master

* refs/pull/35096/head:
cephfs-journal-tool: fix wrong read_offset when get missing objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
4 years agoMerge PR #32120 into master
Patrick Donnelly [Fri, 31 Jul 2020 21:08:41 +0000 (14:08 -0700)]
Merge PR #32120 into master

* refs/pull/32120/head:
client: periodically forward metrics to session MDSs
client: track cap hits and misses globally
client: minor cleanup to MetaSession class

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agotest/librbd: unquiesce notifications are handled lazily 36395/head
Jason Dillaman [Fri, 31 Jul 2020 14:31:07 +0000 (10:31 -0400)]
test/librbd: unquiesce notifications are handled lazily

The quiesce unit tests were expecting that the unquiesce notification
would be processed by the time the snapshot create command completed.
Since the RPC is immediately ACKed, there is a potential for a race
if the test doesn't wait for the unquiesce to process.

Fixes: https://tracker.ceph.com/issues/46679
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #32780 from petrutlucian94/windows.5
Kefu Chai [Fri, 31 Jul 2020 12:36:59 +0000 (20:36 +0800)]
Merge pull request #32780 from petrutlucian94/windows.5

Windows support [part 5]

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoos/bluestore: be more verbose in _open_super_meta by default.
Igor Fedotov [Fri, 11 Oct 2019 14:34:58 +0000 (17:34 +0300)]
os/bluestore: be more verbose in _open_super_meta by default.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoos/bluestore: don't do init_rm_free when mounting bluefs after mkfs.
Igor Fedotov [Fri, 11 Oct 2019 14:15:22 +0000 (17:15 +0300)]
os/bluestore: don't do init_rm_free when mounting bluefs after mkfs.

This is a duplicate op which might cause allocator (namely stupid one) to fail.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoos/bluestore: simlify BlueFS::_stop_alloc
Igor Fedotov [Fri, 11 Oct 2019 13:57:33 +0000 (16:57 +0300)]
os/bluestore: simlify BlueFS::_stop_alloc

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoos/bluestore: fix bluefs migrate/expand to match single allocator.
Igor Fedotov [Thu, 10 Oct 2019 11:25:57 +0000 (14:25 +0300)]
os/bluestore: fix bluefs migrate/expand to match single allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #36065 from trociny/wip-46445
Kefu Chai [Fri, 31 Jul 2020 12:36:25 +0000 (20:36 +0800)]
Merge pull request #36065 from trociny/wip-46445

monclient: schedule first tick using mon_client_hunt_interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/cephadm: Introduce tox and autopep8 36348/head
Volker Theile [Wed, 29 Jul 2020 10:13:01 +0000 (12:13 +0200)]
mgr/cephadm: Introduce tox and autopep8

This PR introduces tox and autopep8 to format the Python code according to PEP8. It does NOT include any lint related things.

Signed-off-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36002 from david-z/wip-fix-slow-req
Kefu Chai [Fri, 31 Jul 2020 12:34:33 +0000 (20:34 +0800)]
Merge pull request #36002 from david-z/wip-fix-slow-req

mgr: don't update osd stat which is already out

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36101 from ifed01/wip-ifed-fix-store-test-with-db
Kefu Chai [Fri, 31 Jul 2020 12:33:24 +0000 (20:33 +0800)]
Merge pull request #36101 from ifed01/wip-ifed-fix-store-test-with-db

test/objectstore_store_test: fix test cases broken due to additional …

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
4 years agoMerge pull request #36337 from tchaikov/wip-qa-ceph-waiting
Kefu Chai [Fri, 31 Jul 2020 12:32:13 +0000 (20:32 +0800)]
Merge pull request #36337 from tchaikov/wip-qa-ceph-waiting

qa/tasks/ceph: do not print out empty list of pg

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36360 from matthewoliver/cephadm_add_ipv6_unwrap
Sebastian Wagner [Fri, 31 Jul 2020 11:31:41 +0000 (13:31 +0200)]
Merge pull request #36360 from matthewoliver/cephadm_add_ipv6_unwrap

cephadm: add unwrap_ipv6 helper method

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36151 from jschmid1/osd_support_reweight_impl
Sebastian Wagner [Fri, 31 Jul 2020 11:28:18 +0000 (13:28 +0200)]
Merge pull request #36151 from jschmid1/osd_support_reweight_impl

mgr/cephadm: rework osd removals/replacements

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36338 from pcuzner/fix-daemon-actions
Sebastian Wagner [Fri, 31 Jul 2020 11:05:18 +0000 (13:05 +0200)]
Merge pull request #36338 from pcuzner/fix-daemon-actions

mgr/cephadm: fix call to cephadm for daemon restarts etc

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36284 from sebastian-philipp/cephadm_service_id_migration
Sebastian Wagner [Fri, 31 Jul 2020 10:54:51 +0000 (12:54 +0200)]
Merge pull request #36284 from sebastian-philipp/cephadm_service_id_migration

mgr/cephadm: Add migration to keep the service names consistent

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36374 from varshar16/wip-doc-orchestrator-update-create-osd
Sebastian Wagner [Fri, 31 Jul 2020 10:41:33 +0000 (12:41 +0200)]
Merge pull request #36374 from varshar16/wip-doc-orchestrator-update-create-osd

doc/mgr/orchestrator: Fix formatting and examples in create OSD

Reviewed-by: Joshua Schmid <jschmid@suse.de>
4 years agoMerge pull request #36376 from tchaikov/wip-crimson-log
Kefu Chai [Fri, 31 Jul 2020 09:45:35 +0000 (17:45 +0800)]
Merge pull request #36376 from tchaikov/wip-crimson-log

qa/tasks/ceph: redirect stderr to log file

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agobuild/debian: remove osd_support 36151/head
Joshua Schmid [Tue, 28 Jul 2020 11:43:11 +0000 (13:43 +0200)]
build/debian: remove osd_support

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agodoc/cephadm: adjust osd removal docs
Joshua Schmid [Tue, 28 Jul 2020 07:35:56 +0000 (09:35 +0200)]
doc/cephadm: adjust osd removal docs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/cephadm: add tests for new remove/replace logic
Joshua Schmid [Fri, 24 Jul 2020 13:25:31 +0000 (15:25 +0200)]
mgr/cephadm: add tests for new remove/replace logic

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/cephadm: implement osd remove/replace logic in cephadm
Joshua Schmid [Fri, 24 Jul 2020 13:24:38 +0000 (15:24 +0200)]
mgr/cephadm: implement osd remove/replace logic in cephadm

Fixes: https://tracker.ceph.com/issues/44548
Fixes: https://tracker.ceph.com/issues/45594
Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/osd_support: remove module and all traces
Joshua Schmid [Fri, 24 Jul 2020 13:23:52 +0000 (15:23 +0200)]
mgr/osd_support: remove module and all traces

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agoMerge pull request #36367 from tchaikov/wip-46771
Kefu Chai [Fri, 31 Jul 2020 07:39:44 +0000 (15:39 +0800)]
Merge pull request #36367 from tchaikov/wip-46771

qa/tasks/ragweed: always set ragweed_repo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agoqa/tasks/ragweed: always set ragweed_repo 36367/head
Kefu Chai [Thu, 30 Jul 2020 08:45:40 +0000 (16:45 +0800)]
qa/tasks/ragweed: always set ragweed_repo

* extract get_ragweed_branch() out of download() task, for better
  readablity.
* use a loop for retry when the first clone fails
* drop the `raise ValueError()` clause as it never happens. we could use
  an assert() here, but i don't think it is necessary anyway.
* use sh() instead of run() for better readablity.
* always set ragweed_repo. before this change this variable is
  unbounded if `force-branch` is set.

Fixes: https://tracker.ceph.com/issues/46771
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/librados_test_stub: handle assert_version 36378/head
Or Ozeri [Thu, 30 Jul 2020 17:15:53 +0000 (20:15 +0300)]
test/librados_test_stub: handle assert_version

This commits adds a proper handling of the assert_version API in RADOS.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
4 years agomgr/cephadm: skip Dashboard configuration if Dashboard is disabled 36386/head
Kiefer Chang [Fri, 31 Jul 2020 02:59:29 +0000 (10:59 +0800)]
mgr/cephadm: skip Dashboard configuration if Dashboard is disabled

Fixes: https://tracker.ceph.com/issues/46045
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
4 years agoMerge pull request #36159 from trociny/wip-46581
Jason Dillaman [Thu, 30 Jul 2020 19:43:20 +0000 (15:43 -0400)]
Merge pull request #36159 from trociny/wip-46581

librbd: handle DNE from immutable-object-cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agomds: place MDSGatherBuilder on the stack 36359/head
Patrick Donnelly [Thu, 30 Jul 2020 02:42:44 +0000 (19:42 -0700)]
mds: place MDSGatherBuilder on the stack

This was not meant to be heap allocated; it's not freed via the Gather
machinery.

Fixes: https://tracker.ceph.com/issues/46766
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36319 from orozery/librados_test_stub_add_objver
Jason Dillaman [Thu, 30 Jul 2020 16:55:15 +0000 (12:55 -0400)]
Merge pull request #36319 from orozery/librados_test_stub_add_objver

test/librados_test_stub: add version tracking of objects

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36317 from orozery/librados_test_stub_exclusive_create
Jason Dillaman [Thu, 30 Jul 2020 16:54:51 +0000 (12:54 -0400)]
Merge pull request #36317 from orozery/librados_test_stub_exclusive_create

test/librados_test_stub: handle exclusive create

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoqa/tasks/ceph: redirect stderr to log file 36376/head
Kefu Chai [Thu, 30 Jul 2020 15:20:35 +0000 (23:20 +0800)]
qa/tasks/ceph: redirect stderr to log file

crimson write log to stderr, let's redirect it to log file for a more
peaceful teuthology.log.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36363 from sebastian-philipp/p-common-pyyaml
Kefu Chai [Thu, 30 Jul 2020 16:20:43 +0000 (00:20 +0800)]
Merge pull request #36363 from sebastian-philipp/p-common-pyyaml

python-common, spec.in: Add missing pyyaml dependency

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36344 from wjwithagen/wjw-fix-_1
Kefu Chai [Thu, 30 Jul 2020 16:09:14 +0000 (00:09 +0800)]
Merge pull request #36344 from wjwithagen/wjw-fix-_1

test: fix ambiguous reference to _1

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agomds: resolve SIGSEGV in waiting for uncommitted fragments 36358/head
Patrick Donnelly [Thu, 30 Jul 2020 02:36:28 +0000 (19:36 -0700)]
mds: resolve SIGSEGV in waiting for uncommitted fragments

The MDSGatherBuilder was not correctly used / wired up.

Fixes: https://tracker.ceph.com/issues/46765
Fixes: 77eb368d2d35f2418875227fff9a34b5ef15a290
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36370 from tchaikov/wip-crimson-scrub2
Kefu Chai [Thu, 30 Jul 2020 15:24:11 +0000 (23:24 +0800)]
Merge pull request #36370 from tchaikov/wip-crimson-scrub2

crimson/osd: handle MOSDScrub2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoos/bluestore: Track per zone metadata 35659/head
Abutalib Aghayev [Tue, 30 Jun 2020 20:33:15 +0000 (16:33 -0400)]
os/bluestore: Track per zone metadata

For every object we maintain <zone_num+oid, offset> tuple in the key-value
store.  When a new object written to a zone, we insert the corresponding tuple
to the database.  When an object is truncated, we remove the corresponding
tuple.  When an object is overwritten, we remove the old tuple and insert a new
tuple corresponding to the new location of the object.  The cleaner can now
identify live objects within the zone <zone_num> by enumerating all the keys
starting with <zone_num> prefix.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
4 years agocephadm: Fix error bootstraping with '--registry-json' 36375/head
Ricardo Marques [Thu, 30 Jul 2020 14:48:21 +0000 (15:48 +0100)]
cephadm: Fix error bootstraping with '--registry-json'

We need to force setting registry auth options because
these options are unknown before the mgr first starts
and registers them.

Fixes: https://tracker.ceph.com/issues/46777
Signed-off-by: Ricardo Marques <rimarques@suse.com>
4 years agoMerge pull request #36286 from sebastian-philipp/cephadm-notify_config-ceph-conf...
Sebastian Wagner [Thu, 30 Jul 2020 14:20:44 +0000 (16:20 +0200)]
Merge pull request #36286 from sebastian-philipp/cephadm-notify_config-ceph-conf-race

mgr/cephadm: revamp ceph.conf distribution scheduling

Reviewed-by: Ricardo Marques <rimarques@suse.com>
4 years agoMerge pull request #36231 from tspmelo/wip-fix-overflow
Kiefer Chang [Thu, 30 Jul 2020 14:02:37 +0000 (22:02 +0800)]
Merge pull request #36231 from tspmelo/wip-fix-overflow

mgr/dashboard: Configure overflow of popover in health page

Reviewed-by: Ni-Feng Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agodoc/mgr/orchestrator: Fix formatting and examples in create OSDs 36374/head
Varsha Rao [Thu, 30 Jul 2020 13:28:47 +0000 (15:28 +0200)]
doc/mgr/orchestrator: Fix formatting and examples in create OSDs

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36243 from Devp00l/wip-46660
Volker Theile [Thu, 30 Jul 2020 13:00:59 +0000 (15:00 +0200)]
Merge pull request #36243 from Devp00l/wip-46660

mgr/dashboard: Fix regression on table error handling

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agomgr/cephadm: Add migration to keep the service names consistent 36284/head
Sebastian Wagner [Fri, 24 Jul 2020 14:20:22 +0000 (16:20 +0200)]
mgr/cephadm: Add migration to keep the service names consistent

After 15.2.4, we unified some service IDs: MONs, MGRs etc no longer have a service id.
Which means, the service names changed:

mon.foo -> mon
mgr.foo -> mgr

This fixes the data structure consistency

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: minor cleanup in test_migration
Sebastian Wagner [Fri, 24 Jul 2020 14:17:45 +0000 (16:17 +0200)]
mgr/cephadm: minor cleanup in test_migration

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36324 from dillaman/wip-46737
Mykola Golub [Thu, 30 Jul 2020 12:04:10 +0000 (15:04 +0300)]
Merge pull request #36324 from dillaman/wip-46737

librbd: ensure image cannot be closed until in-flight IO callbacks complete

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agocrimson/osd: handle MOSDScrub2 36370/head
Kefu Chai [Thu, 30 Jul 2020 11:45:23 +0000 (19:45 +0800)]
crimson/osd: handle MOSDScrub2

MOSDScrub2 is sent from mgr for serving "ceph pg
{scrub|deep-scrub|repair}' commands when it's talking to a mimic and newer OSD.

ceph task checks if all pgs are scrubbed by looking at the `last_scrub_stamp` fields
in the `ceph pg dump` output. and it request the not-yet-scrubbed pgs a
deep scrub to ensure they are scrub before timeout.

in this change, crimson handles MOSDScrub2 by starting a remote peering
request, and the underlying peering_state will notify the corresponding
PG to start scrub. to get the test pass, a minimal implmentation is
added to update the scrub timestamp to `now` upon request of
peering_state.

we will need to add the correct scrubbing support later. but this is
enough for passing the thrasher test and for preparing for more tests
which uses the "ceph" task.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36281 from sebastian-philipp/mgr-tox-rm-cov
Sebastian Wagner [Thu, 30 Jul 2020 11:42:53 +0000 (13:42 +0200)]
Merge pull request #36281 from sebastian-philipp/mgr-tox-rm-cov

pybind/mgr: remove coverage from tox.ini

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36335 from mgfritch/cephadm-vstart-mon-cidr
Sebastian Wagner [Thu, 30 Jul 2020 10:27:08 +0000 (12:27 +0200)]
Merge pull request #36335 from mgfritch/cephadm-vstart-mon-cidr

vstart: infer the mon public_network

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #36334 from mgfritch/cephadm-event-multiline
Sebastian Wagner [Thu, 30 Jul 2020 10:13:05 +0000 (12:13 +0200)]
Merge pull request #36334 from mgfritch/cephadm-event-multiline

mgr/orch: allow for multiline OrchestratorEvent message

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36162 from Daniel-Pivonka/cephadm-43681
Sebastian Wagner [Thu, 30 Jul 2020 09:55:33 +0000 (11:55 +0200)]
Merge pull request #36162 from Daniel-Pivonka/cephadm-43681

mgr/cephadm: streamline rgw deployment

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agotest: fix ambiguous reference to _1 36344/head
Willem Jan Withagen [Wed, 29 Jul 2020 12:01:10 +0000 (12:01 +0000)]
test: fix ambiguous reference to _1

FreeBSD/Clang/lib complains:
/home/jenkins/workspace/ceph-master/src/test/librados_test_stub/TestRadosClient.cc:295:55: error: reference to '_1' is ambiguous
    &TestRadosClient::finish_aio_completion, this, c, _1));
                                                      ^
/usr/local/include/boost/bind/placeholders.hpp:46:38: note: candidate found by name lookup is 'boost::placeholders::_1'
BOOST_STATIC_CONSTEXPR boost::arg<1> _1;
                                     ^
/usr/include/c++/v1/functional:2448:46: note: candidate found by name lookup is 'std::__1::placeholders::_1'
/* _LIBCPP_INLINE_VAR */ constexpr __ph<1>   _1{};

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #36362 from tchaikov/wip-crimson-read-write-file
Kefu Chai [Thu, 30 Jul 2020 09:16:14 +0000 (17:16 +0800)]
Merge pull request #36362 from tchaikov/wip-crimson-read-write-file

crimson: extract read_file()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agopython-common, spec.in: Add missing pyyaml dependency 36363/head
Sebastian Wagner [Thu, 30 Jul 2020 07:55:08 +0000 (09:55 +0200)]
python-common, spec.in: Add missing pyyaml dependency

Fixes: cd6a488ab2ca036dd4fb36751b938f605e97e1c8
Fixes: https://tracker.ceph.com/issues/46759
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoclient: clean up the unnecessary client_lock for _conf->client_trace 36364/head
Xiubo Li [Wed, 29 Jul 2020 06:58:31 +0000 (14:58 +0800)]
client: clean up the unnecessary client_lock for _conf->client_trace

There is no need to make the "cct->_conf->client_trace" under the
client_lock, it is for the "ceph-syn" and it almost won't change,
even it will but it still won't cause any race.

For the "inode->ino" and "inode->snapid", since they won't change
once initialized when creating the inode, and the inode here will
be protected by the reference count, so no need the client_lock too.

Fixes: https://tracker.ceph.com/issues/46768
Signed-off-by: Xiubo Li <xiubli@redhat.com>