]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoqa: add debugging failed osd-release setting
Patrick Donnelly [Fri, 16 Aug 2019 21:54:48 +0000 (14:54 -0700)]
qa: add debugging failed osd-release setting

See-also: https://tracker.ceph.com/issues/40773
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29544 from tchaikov/wip-doc-search-CSP
Kefu Chai [Thu, 22 Aug 2019 09:56:58 +0000 (17:56 +0800)]
Merge pull request #29544 from tchaikov/wip-doc-search-CSP

doc: always load resources via HTTPS

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agodoc: always load resources via HTTPS 29544/head
Kefu Chai [Thu, 8 Aug 2019 10:40:47 +0000 (18:40 +0800)]
doc: always load resources via HTTPS

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29755 from xiexingguo/wip-inc-recovery-4
Xie Xingguo [Thu, 22 Aug 2019 05:48:15 +0000 (13:48 +0800)]
Merge pull request #29755 from xiexingguo/wip-inc-recovery-4

osd: do not invalidate clear_regions of missing item at boot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #29804 from alfredodeza/wip-rm41378
Andrew Schoen [Wed, 21 Aug 2019 21:59:18 +0000 (16:59 -0500)]
Merge pull request #29804 from alfredodeza/wip-rm41378

ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge PR #29744 into master
Sage Weil [Wed, 21 Aug 2019 20:02:27 +0000 (15:02 -0500)]
Merge PR #29744 into master

* refs/pull/29744/head:
qa/run-standalone.sh: fix python path
qa/standalone/mon/health-mute.sh: fix up rachet test
qa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge PR #29749 into master
Sage Weil [Wed, 21 Aug 2019 20:02:14 +0000 (15:02 -0500)]
Merge PR #29749 into master

* refs/pull/29749/head:
mon/HealthMonitor: remove unused label

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29757 into master
Sage Weil [Wed, 21 Aug 2019 20:02:01 +0000 (15:02 -0500)]
Merge PR #29757 into master

* refs/pull/29757/head:
osd: always initialize local variable

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29763 into master
Sage Weil [Wed, 21 Aug 2019 20:01:50 +0000 (15:01 -0500)]
Merge PR #29763 into master

* refs/pull/29763/head:
qa/suites/rados: whitelist POOL_APP_NOT_ENABLED warning

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt 29804/head
Alfredo Deza [Wed, 21 Aug 2019 18:15:32 +0000 (14:15 -0400)]
ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PR #28378 into master
Patrick Donnelly [Wed, 21 Aug 2019 17:57:15 +0000 (10:57 -0700)]
Merge PR #28378 into master

* refs/pull/28378/head:
qa/tasks: introduce Thrasher base class
qa/tasks: Fix typo
qa/tasks: manage thrashers
qa/tasks: start DaemonWatchdog when ceph starts
qa/tasks: make watch and bark handle more daemons
qa/tasks: move DaemonWatchdog to new file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29773 from dillaman/wip-41352
Mykola Golub [Wed, 21 Aug 2019 14:26:01 +0000 (17:26 +0300)]
Merge pull request #29773 from dillaman/wip-41352

pybind/mgr/rbd_support: fix missing variable in error path

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #29659 from jan--f/c-v-simple-functional-no-lvm-zap
Jan Fajerski [Wed, 21 Aug 2019 07:17:30 +0000 (09:17 +0200)]
Merge pull request #29659 from jan--f/c-v-simple-functional-no-lvm-zap

ceph-volume: don't try to test lvm zap on simple tests

6 years agoqa/tasks: introduce Thrasher base class 28378/head
Jos Collin [Mon, 5 Aug 2019 10:52:10 +0000 (16:22 +0530)]
qa/tasks: introduce Thrasher base class

* Introduced a Thrasher base class.
* Updated thrashers to inherit from Thrasher.
* Replaced the magic variable e with Thrasher.exception as per the discussion.
  Now the exception variable sets by default as the thrashers are inheriting
  from the Thrasher class.

Fixes: https://github.com/ceph/ceph/pull/28378#discussion_r309337928
Fixes: https://tracker.ceph.com/issues/41133
Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agomon/HealthMonitor: remove unused label 29749/head
Kefu Chai [Tue, 20 Aug 2019 02:05:09 +0000 (10:05 +0800)]
mon/HealthMonitor: remove unused label

move the whole sanity checks into `HealthMonitor::preprocess_command()`.

this change silences warning of:

warning: label 'reply' defined but not used [-Wunused-label]

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #29717 into master
Sage Weil [Tue, 20 Aug 2019 21:39:28 +0000 (16:39 -0500)]
Merge PR #29717 into master

* refs/pull/29717/head:
os/bluestore: prefix omap of temp objects by real pool

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29759 into master
Patrick Donnelly [Tue, 20 Aug 2019 17:24:08 +0000 (10:24 -0700)]
Merge PR #29759 into master

* refs/pull/29759/head:
qa/tasks: fixed typo in the comment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29625 from kamoltat/wip-test-progress-fix-bug-#29385
Kefu Chai [Tue, 20 Aug 2019 15:09:58 +0000 (23:09 +0800)]
Merge pull request #29625 from kamoltat/wip-test-progress-fix-bug-#29385

mgr/progress/module.py: s/events/_events/

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29704 from tchaikov/wip-ceph-objectstore-tool
Kefu Chai [Tue, 20 Aug 2019 15:07:55 +0000 (23:07 +0800)]
Merge pull request #29704 from tchaikov/wip-ceph-objectstore-tool

ceph-objectstore-tool: return 0 if incmap is sane

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #29655 from trociny/wip-40923
Jason Dillaman [Tue, 20 Aug 2019 15:04:04 +0000 (11:04 -0400)]
Merge pull request #29655 from trociny/wip-40923

qa/workunits/rbd: stress test `rbd mirror pool status --verbose`

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agopybind/mgr/rbd_support: fix missing variable in error path 29773/head
Jason Dillaman [Tue, 20 Aug 2019 15:00:08 +0000 (11:00 -0400)]
pybind/mgr/rbd_support: fix missing variable in error path

Fixes: https://tracker.ceph.com/issues/41352
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #29376 from Songweibin/wip-rbd-display-id
Jason Dillaman [Tue, 20 Aug 2019 14:23:36 +0000 (10:23 -0400)]
Merge pull request #29376 from Songweibin/wip-rbd-display-id

rbd/action: display image id in rbd du/list output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #29743 from smithfarm/wip-ceph-backport-https
Nathan Cutler [Tue, 20 Aug 2019 14:23:06 +0000 (16:23 +0200)]
Merge pull request #29743 from smithfarm/wip-ceph-backport-https

script/ceph-backport.sh: carry https through to logical conclusion

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29653 from lixiaoy1/AsyncOpTracker_dup
Jason Dillaman [Tue, 20 Aug 2019 14:22:38 +0000 (10:22 -0400)]
Merge pull request #29653 from lixiaoy1/AsyncOpTracker_dup

librbd: Remove duplicated AsyncOpTracker in librbd/Utils.h

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #29094 from cbodley/wip-40806
Casey Bodley [Tue, 20 Aug 2019 14:19:07 +0000 (10:19 -0400)]
Merge pull request #29094 from cbodley/wip-40806

radosgw-admin: bucket sync status not 'caught up' during full sync

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
6 years agoscripts/ceph-backport.sh: always use https://tracker.ceph.com 29743/head
Nathan Cutler [Mon, 19 Aug 2019 14:57:07 +0000 (16:57 +0200)]
scripts/ceph-backport.sh: always use https://tracker.ceph.com

Completing the wave of fixes to this script in the wake of
https://tracker.ceph.com/issues/38764, this commit replaces
"http" with "https" in the comments and puts the Redmine endpoint
into a variable, along with some other cleanups.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #29762 from alfredodeza/bz-1738379
Alfredo Deza [Tue, 20 Aug 2019 13:32:05 +0000 (09:32 -0400)]
Merge pull request #29762 from alfredodeza/bz-1738379

ceph-volume: use the OSD identifier when reporting success

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoqa/suites/rados: whitelist POOL_APP_NOT_ENABLED warning 29763/head
Kefu Chai [Tue, 20 Aug 2019 13:05:21 +0000 (21:05 +0800)]
qa/suites/rados: whitelist POOL_APP_NOT_ENABLED warning

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: login screen language dropdown enhancement (#29636)
Lenz Grimmer [Tue, 20 Aug 2019 12:48:49 +0000 (12:48 +0000)]
mgr/dashboard: login screen language dropdown enhancement (#29636)

mgr/dashboard: login screen language dropdown enhancement

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #29737 from tspmelo/wip-label-alignment
Lenz Grimmer [Tue, 20 Aug 2019 12:20:05 +0000 (12:20 +0000)]
Merge pull request #29737 from tspmelo/wip-label-alignment

mgr/dashboard: Vertically align the "Refresh" label

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: Exclude some folders in the frontend project (#29708)
Lenz Grimmer [Tue, 20 Aug 2019 12:18:53 +0000 (12:18 +0000)]
mgr/dashboard: Exclude some folders in the frontend project (#29708)

mgr/dashboard: Exclude some folders in the frontend project

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoceph-volume devices.lvm zap use the identifier to report success 29762/head
Alfredo Deza [Tue, 20 Aug 2019 12:04:56 +0000 (08:04 -0400)]
ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume tests check success message when zapping
Alfredo Deza [Tue, 20 Aug 2019 12:04:26 +0000 (08:04 -0400)]
ceph-volume tests check success message when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agomgr/dashboard: Hosts Page Service Links Test (#29516)
Lenz Grimmer [Tue, 20 Aug 2019 10:43:53 +0000 (10:43 +0000)]
mgr/dashboard: Hosts Page Service Links Test (#29516)

mgr/dashboard: Hosts Page Service Links Test

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29728 from tchaikov/wip-41330
Kefu Chai [Tue, 20 Aug 2019 10:31:18 +0000 (18:31 +0800)]
Merge pull request #29728 from tchaikov/wip-41330

cmake,run-make-check.sh: disable SPDK by default

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/tasks: fixed typo in the comment 29759/head
Jos Collin [Tue, 20 Aug 2019 09:49:55 +0000 (15:19 +0530)]
qa/tasks: fixed typo in the comment

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agoosd: always initialize local variable 29757/head
Kefu Chai [Tue, 20 Aug 2019 08:15:17 +0000 (16:15 +0800)]
osd: always initialize local variable

to silence a GCC warning like:

../src/osd/OSD.cc:4608:24: warning: ‘sender_delta_ub’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
 4608 |       ceph::signedspan sender_delta_ub;
      |                        ^~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake,run-make-check.sh,deb,rpm: disable SPDK by default 29728/head
Kefu Chai [Mon, 19 Aug 2019 07:21:06 +0000 (15:21 +0800)]
cmake,run-make-check.sh,deb,rpm: disable SPDK by default

but we still enable it in `run-make-check.sh`

* cmake: disable SPDK by default
* run-make-check.sh: enable WITH_SPDK so at least we can ensure it
  builds
* deb,rpm: add uuid-dev / libuuid-devel as a "make check" dependency

Fixes: https://tracker.ceph.com/issues/41330
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: make WITH_SPDK a dependent option
Kefu Chai [Mon, 19 Aug 2019 07:18:49 +0000 (15:18 +0800)]
cmake: make WITH_SPDK a dependent option

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: do not invalidate clear_regions of missing item at boot 29755/head
xie xingguo [Mon, 17 Jun 2019 03:05:31 +0000 (11:05 +0800)]
osd: do not invalidate clear_regions of missing item at boot

Seems we'll always mark clear_regions as all dirty
when reading pg logs and missing items off the disk,
which as a result turns incremental recovery off by default.
Also using std::move seems to be a bit more efficient
and robust here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #29753 from tchaikov/wip-sudo-sysctl
Kefu Chai [Tue, 20 Aug 2019 03:57:03 +0000 (11:57 +0800)]
Merge pull request #29753 from tchaikov/wip-sudo-sysctl

run-make-check.sh: run sudo with absolute path

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #29726 from xiexingguo/wip-inc-recovery-1
Xie Xingguo [Tue, 20 Aug 2019 03:22:39 +0000 (11:22 +0800)]
Merge pull request #29726 from xiexingguo/wip-inc-recovery-1

osd/PrimaryLogPG: fix dirty range of write_full

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29729 from xiexingguo/wip-inc-recovery-2
Xie Xingguo [Tue, 20 Aug 2019 03:22:08 +0000 (11:22 +0800)]
Merge pull request #29729 from xiexingguo/wip-inc-recovery-2

osd/PrimaryLogPG: finish_copyfrom - dirty omap if necessary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agorun-make-check.sh: run sudo with absolute path 29753/head
Kefu Chai [Tue, 20 Aug 2019 02:49:18 +0000 (10:49 +0800)]
run-make-check.sh: run sudo with absolute path

there is chance that `/sbin` is not listed in `secure_path`, see
sudoers(5) for more details

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/run-standalone.sh: fix python path 29744/head
Sage Weil [Mon, 19 Aug 2019 17:33:05 +0000 (12:33 -0500)]
qa/run-standalone.sh: fix python path

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/mon/health-mute.sh: fix up rachet test
Sage Weil [Mon, 19 Aug 2019 17:28:31 +0000 (12:28 -0500)]
qa/standalone/mon/health-mute.sh: fix up rachet test

Make sure we provide time for the mute to get cleared out by tick().

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29518 into master
Patrick Donnelly [Mon, 19 Aug 2019 16:36:37 +0000 (09:36 -0700)]
Merge PR #29518 into master

* refs/pull/29518/head:
mount.ceph: properly handle -o strictatime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29411 into master
Patrick Donnelly [Mon, 19 Aug 2019 16:34:42 +0000 (09:34 -0700)]
Merge PR #29411 into master

* refs/pull/29411/head:
mds: fix InoTable::force_consume_to()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29543 into master
Patrick Donnelly [Mon, 19 Aug 2019 16:33:26 +0000 (09:33 -0700)]
Merge PR #29543 into master

* refs/pull/29543/head:
mds: move some MDCache member init to header

Reviewed-by: Jos Collin <jcollin@redhat.com>
6 years agoMerge pull request #29429 from dillaman/wip-40834
Jason Dillaman [Mon, 19 Aug 2019 15:57:01 +0000 (11:57 -0400)]
Merge pull request #29429 from dillaman/wip-40834

doc/rbd: initial kubernetes / ceph-csi integration documentation

Reviewed-by: ShyamsundarR <srangana@redhat.com>
Reviewed-by: Humble Chirammal <hchiramm@redhat.com>
Reviewed-by: Madhu Rajanna <madhupr007@gmail.com>
6 years agoqa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/
Sage Weil [Mon, 19 Aug 2019 14:27:51 +0000 (09:27 -0500)]
qa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29719 into master
Sage Weil [Mon, 19 Aug 2019 14:09:06 +0000 (09:09 -0500)]
Merge PR #29719 into master

* refs/pull/29719/head:
osd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: E2E Test for RBD Mirroring and Images (#29381)
Lenz Grimmer [Mon, 19 Aug 2019 13:10:59 +0000 (13:10 +0000)]
mgr/dashboard: E2E Test for RBD Mirroring and Images (#29381)

mgr/dashboard: E2E Test for RBD Mirroring and Images

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agomgr/dashboard: Vertically align the "Refresh" label 29737/head
Tiago Melo [Mon, 19 Aug 2019 10:59:53 +0000 (10:59 +0000)]
mgr/dashboard: Vertically align the "Refresh" label

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agomgr/dashboard: Exclude some folders in the frontend project 29708/head
Tiago Melo [Thu, 8 Aug 2019 12:58:39 +0000 (12:58 +0000)]
mgr/dashboard: Exclude some folders in the frontend project

This will remove an warning in VS Code [1] and, more important, will reduce the
number of files that TS will watch when compiling or linting.

[1] To enable project-wide JavaScript/TypeScript language features,
exclude large folders with source files that you do not work on.

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29685 from smithfarm/wip-41296
Nathan Cutler [Mon, 19 Aug 2019 10:17:46 +0000 (12:17 +0200)]
Merge pull request #29685 from smithfarm/wip-41296

rpm: always build ceph-test package

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
6 years agoMerge pull request #29556 from Devp00l/wip-41165
Lenz Grimmer [Mon, 19 Aug 2019 09:08:43 +0000 (09:08 +0000)]
Merge pull request #29556 from Devp00l/wip-41165

mgr/dashboard: CephFS client tab switch

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: Verify fields on ISCSI page (#29608)
Lenz Grimmer [Mon, 19 Aug 2019 09:03:57 +0000 (09:03 +0000)]
mgr/dashboard: Verify fields on ISCSI page (#29608)

mgr/dashboard: Verify fields on ISCSI page

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: clean-up python unit tests (#28696)
Lenz Grimmer [Mon, 19 Aug 2019 08:52:53 +0000 (08:52 +0000)]
mgr/dashboard: clean-up python unit tests (#28696)

mgr/dashboard: clean-up python unit tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agoosd/PrimaryLogPG: finish_copyfrom - dirty omap if necessary 29729/head
xie xingguo [Wed, 14 Aug 2019 07:37:49 +0000 (15:37 +0800)]
osd/PrimaryLogPG: finish_copyfrom - dirty omap if necessary

otherwise we'll at risk of losing track of omap entries.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoosd/PrimaryLogPG: fix dirty range of write_full 29726/head
xie xingguo [Wed, 14 Aug 2019 06:12:43 +0000 (14:12 +0800)]
osd/PrimaryLogPG: fix dirty range of write_full

A write_full operation may implicitly truncate the object down,
hence we need to mark the truncated part as dirty as well since
follow-up randomized writes may still be able to (re)extend the
object size and leave some holes against the truncated part,
which as a result might cause problems during incremental-mode
recovery.

Note that write_update_size_and_usage would reset oi.size
and that's why we move the mark_data_region_dirty call before that.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #29584 from tchaikov/wip-qa-test_envlibrados_for_rocksdb-cmake3
Kefu Chai [Sun, 18 Aug 2019 04:03:56 +0000 (12:03 +0800)]
Merge pull request #29584 from tchaikov/wip-qa-test_envlibrados_for_rocksdb-cmake3

qa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoqa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake 29584/head
Kefu Chai [Sat, 10 Aug 2019 08:08:00 +0000 (16:08 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake

rocksdb needs CMake 3.5.1 since
https://github.com/facebook/rocksdb/pull/5662

Fixes: https://tracker.ceph.com/issues/41253
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments 29719/head
xie xingguo [Sat, 17 Aug 2019 07:19:46 +0000 (15:19 +0800)]
osd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments

The ondisk_{read,write}_lock infrastructure was long gone with
https://github.com/ceph/ceph/pull/20177 merged - c244300ef33a044ad71fea7d92d77f33b5d41851,
to be specific. Hence the related comments must die since they
could be super-misleading.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #29433 from dillaman/wip-41029
Mykola Golub [Sat, 17 Aug 2019 06:27:02 +0000 (09:27 +0300)]
Merge pull request #29433 from dillaman/wip-41029

pybind/mgr/rbd_support: ignore missing support for RBD namespaces

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoos/bluestore: prefix omap of temp objects by real pool 29717/head
xie xingguo [Fri, 16 Aug 2019 07:28:54 +0000 (15:28 +0800)]
os/bluestore: prefix omap of temp objects by real pool

For recovery or backfill, we use temp object at destination
if the whole object context can not sent in one shot.
And since https://github.com/ceph/ceph/pull/29292, we now segregate
omap keys by object's binding pool.
However, https://github.com/ceph/ceph/pull/29292 does not
work well for recovery or backfill process that need
to manipulate temp objects because we always (deliberately)
assign a negative pool id for each temp object which is
(obviously) different from the corresponding target object,
and we do not fix it when trying to rename the temp object
back to the target object at the end of recovery/backfill,
as a result of which we totally lose track of the omap
portion of the recovered object.

Fix by prefixing all omap-related stuff of temp objects
by using its real(pg's) pool.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomds: move some MDCache member init to header 29543/head
Patrick Donnelly [Wed, 7 Aug 2019 16:56:44 +0000 (09:56 -0700)]
mds: move some MDCache member init to header

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29712 from neha-ojha/wip-basic-workflow
Neha Ojha [Fri, 16 Aug 2019 21:22:06 +0000 (14:22 -0700)]
Merge pull request #29712 from neha-ojha/wip-basic-workflow

doc/dev/developer_guide/index.rst: add youtube reference for Getting Started

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agomgr/progress/module.py: s/events/_events/ #29625 29625/head
Kamoltat (Junior) Sirivadhna [Tue, 13 Aug 2019 03:15:25 +0000 (23:15 -0400)]
mgr/progress/module.py: s/events/_events/ #29625

Fixes time out failure because the module was trying to accessing
attribute events line 437 in this block of code:

435 if marked == "in":
436 for ev_id in list(self._events):
437 ev = self.events[ev_id]
438 if isinstance(ev, PgRecoveryEvent) and osd_id in ev.which_osds:
439 self.log.info("osd.{0} came back in, cancelling event".format(
440 osd_id
441 ))
442 self._complete(ev)

Therefore, it wouldn't trigger an event

Fixes: https://tracker.ceph.com/issues/40618
Signed-off-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
6 years agodoc/dev/developer_guide/index.rst: add youtube ref for Getting Started 29712/head
Neha Ojha [Fri, 16 Aug 2019 19:46:57 +0000 (12:46 -0700)]
doc/dev/developer_guide/index.rst: add youtube ref for Getting Started

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29448 from cbodley/wip-41045
Casey Bodley [Fri, 16 Aug 2019 19:52:11 +0000 (15:52 -0400)]
Merge pull request #29448 from cbodley/wip-41045

rgw: datalog/mdlog trim commands loop until done

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agomgr/dashboard: clean-up tox and unit tests 28696/head
Ernesto Puerta [Fri, 19 Jul 2019 16:10:49 +0000 (18:10 +0200)]
mgr/dashboard: clean-up tox and unit tests

Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.

Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)

Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst

Doc:
- Update HACKING.rst

Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.

Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #29548 into master
Sage Weil [Fri, 16 Aug 2019 16:39:13 +0000 (11:39 -0500)]
Merge PR #29548 into master

* refs/pull/29548/head:
osd: add device_id to list_devices to help get smart info easily

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #29709 from rzarzynski/wip-osd-no-mds-pglsparent
Kefu Chai [Fri, 16 Aug 2019 15:29:07 +0000 (23:29 +0800)]
Merge pull request #29709 from rzarzynski/wip-osd-no-mds-pglsparent

osd: remove orphan include after PGLSParentFilter.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoradosgw-admin: 'mdlog trim' loops until done 29448/head
Casey Bodley [Wed, 31 Jul 2019 21:06:13 +0000 (17:06 -0400)]
radosgw-admin: 'mdlog trim' loops until done

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoradosgw-admin: 'datalog trim' takes shard-id and loops until done
Casey Bodley [Wed, 31 Jul 2019 21:05:40 +0000 (17:05 -0400)]
radosgw-admin: 'datalog trim' takes shard-id and loops until done

Fixes: https://tracker.ceph.com/issues/41045
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29695 from cbodley/wip-41300
Casey Bodley [Fri, 16 Aug 2019 15:21:12 +0000 (11:21 -0400)]
Merge pull request #29695 from cbodley/wip-41300

rgw: RGWSI_User_Module filters '.buckets' objects out of user listing

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoMerge pull request #26787 from soumyakoduri/bucket_name_validation
Casey Bodley [Fri, 16 Aug 2019 14:53:54 +0000 (10:53 -0400)]
Merge pull request #26787 from soumyakoduri/bucket_name_validation

[rgw]:Validate bucket names as per revised s3 spec

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29515 from ivancich/wip-clean-up-reset-stats
Casey Bodley [Fri, 16 Aug 2019 14:21:02 +0000 (10:21 -0400)]
Merge pull request #29515 from ivancich/wip-clean-up-reset-stats

rgw: housekeeping of reset stats operation in radosgw-admin and cls back-end

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #28904 from ZVampirEM77/wip-s3website-fix
Casey Bodley [Fri, 16 Aug 2019 14:20:24 +0000 (10:20 -0400)]
Merge pull request #28904 from ZVampirEM77/wip-s3website-fix

rgw: fix the bug of rgw not doing necessary checking to website configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29215 from tianshan/fix_list_with_delimiter
Casey Bodley [Fri, 16 Aug 2019 14:19:50 +0000 (10:19 -0400)]
Merge pull request #29215 from tianshan/fix_list_with_delimiter

rgw: fix list bucket with delimiter wrongly skip some special keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #29369 from tianshan/fix_realm_watch
Casey Bodley [Fri, 16 Aug 2019 14:18:27 +0000 (10:18 -0400)]
Merge pull request #29369 from tianshan/fix_realm_watch

rgw: fix potential realm watch lost

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29380 from theanalyst/rgw-dns-name-fixes
Casey Bodley [Fri, 16 Aug 2019 14:18:08 +0000 (10:18 -0400)]
Merge pull request #29380 from theanalyst/rgw-dns-name-fixes

rgw: make dns hostnames matching case insensitive

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28558 from cbodley/wip-rgw-bucket-max-aio
Casey Bodley [Fri, 16 Aug 2019 14:17:33 +0000 (10:17 -0400)]
Merge pull request #28558 from cbodley/wip-rgw-bucket-max-aio

[RFC] rgw: raise default rgw_bucket_index_max_aio to 128

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoosd: remove orphan include after PGLSParentFilter. 29709/head
Radoslaw Zarzynski [Fri, 16 Aug 2019 14:15:34 +0000 (16:15 +0200)]
osd: remove orphan include after PGLSParentFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #28824 from dang/zipper
Casey Bodley [Fri, 16 Aug 2019 13:49:28 +0000 (09:49 -0400)]
Merge pull request #28824 from dang/zipper

rgw: Project Zipper part 1

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agomgr/dashboard: clean-up python unit tests
Ernesto Puerta [Fri, 19 Jul 2019 15:23:56 +0000 (17:23 +0200)]
mgr/dashboard: clean-up python unit tests

Fix pylint issues.

Refactor JwtManager check to avoid code duplication (found by pylint
when --jobs=1).

Fix issue with DashboardException.code property, using abs() on
potentially None attribute.

Disables Doctests in services/rbd.py that are actually integration tests
(they check the value of rbd.RBD_FEATURES_NAME_MAPPING). Ideally, these
kind of tests should be explicitly executed in an integration testing
stage, rather that unit-testing. Disabled tests have been prepended with
@DISABLEDOCTEST token.

Fixes: https://tracker.ceph.com/issues/40487
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #29621 from xiexingguo/wip-build-push-segv
Kefu Chai [Fri, 16 Aug 2019 12:25:55 +0000 (20:25 +0800)]
Merge pull request #29621 from xiexingguo/wip-build-push-segv

osd/ReplicatedBackend: check against empty data_included before enabling crc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #29579 from liewegas/wip-big-vs-bluestore
Kefu Chai [Fri, 16 Aug 2019 12:24:43 +0000 (20:24 +0800)]
Merge pull request #29579 from liewegas/wip-big-vs-bluestore

osd: scrub error on big objects; make bluestore refuse to start on big objects

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29663 from cxytt/bugfix
Kefu Chai [Fri, 16 Aug 2019 12:22:32 +0000 (20:22 +0800)]
Merge pull request #29663 from cxytt/bugfix

osd/PrimaryLogPG: Avoid accessing destroyed references in finish_degr…

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29646 from zy751713126/get_pool_name_and_id
Kefu Chai [Fri, 16 Aug 2019 12:21:34 +0000 (20:21 +0800)]
Merge pull request #29646 from zy751713126/get_pool_name_and_id

pybind/rados: add Ioctx.get_pool_id() and Ioctx.get_pool_name()

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29675 from tchaikov/wip-remove-parent-pgls-filter
Kefu Chai [Fri, 16 Aug 2019 12:05:00 +0000 (20:05 +0800)]
Merge pull request #29675 from tchaikov/wip-remove-parent-pgls-filter

osd/PrimaryLogPG: remove unused "parent" pgls-filter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoqa/workunits/rbd: stress test `rbd mirror pool status --verbose` 29655/head
Mykola Golub [Wed, 14 Aug 2019 07:40:02 +0000 (08:40 +0100)]
qa/workunits/rbd: stress test `rbd mirror pool status --verbose`

Fixes: https://tracker.ceph.com/issues/40923
Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #29597 from tchaikov/wip-qa/tasks/cbt
Kefu Chai [Fri, 16 Aug 2019 11:17:58 +0000 (19:17 +0800)]
Merge pull request #29597 from tchaikov/wip-qa/tasks/cbt

qa/tasks/cbt.py: use "git --depth 1 for" faster clone

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoceph-objectstore-tool: return 0 if incmap is sane 29704/head
Kefu Chai [Fri, 16 Aug 2019 11:09:24 +0000 (19:09 +0800)]
ceph-objectstore-tool: return 0 if incmap is sane

this change addresses the regression introduced by 2e2414b3

Fixes: https://tracker.ceph.com/issues/41177
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29683 from jan--f/c-v-keep-device-list-as-lists
Alfredo Deza [Fri, 16 Aug 2019 11:00:50 +0000 (07:00 -0400)]
Merge pull request #29683 from jan--f/c-v-keep-device-list-as-lists

ceph-volume: don't keep device lists as sets

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PR #29676 into master
Sage Weil [Thu, 15 Aug 2019 19:03:37 +0000 (14:03 -0500)]
Merge PR #29676 into master

* refs/pull/29676/head:
test/unittest_bluefs: always remove temp bdev file

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29581 into master
Sage Weil [Thu, 15 Aug 2019 19:02:34 +0000 (14:02 -0500)]
Merge PR #29581 into master

* refs/pull/29581/head:
os/bluestore: do not set osd_memory_target default from cgroup limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
6 years agoMerge PR #29577 into master
Sage Weil [Thu, 15 Aug 2019 19:02:00 +0000 (14:02 -0500)]
Merge PR #29577 into master

* refs/pull/29577/head:
os/bluestore/KernelDevice: fix RW_IO_MAX constant
os/bluestore/KernelDevice: print aio error extent in hex

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29578 from theanalyst/rgw-user-policy-urlencode
Casey Bodley [Thu, 15 Aug 2019 18:02:03 +0000 (14:02 -0400)]
Merge pull request #29578 from theanalyst/rgw-user-policy-urlencode

rgw: url decode PutUserPolicy params

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>