]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoqa/run-standalone.sh: fix python path 29744/head
Sage Weil [Mon, 19 Aug 2019 17:33:05 +0000 (12:33 -0500)]
qa/run-standalone.sh: fix python path

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/mon/health-mute.sh: fix up rachet test
Sage Weil [Mon, 19 Aug 2019 17:28:31 +0000 (12:28 -0500)]
qa/standalone/mon/health-mute.sh: fix up rachet test

Make sure we provide time for the mute to get cleared out by tick().

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/
Sage Weil [Mon, 19 Aug 2019 14:27:51 +0000 (09:27 -0500)]
qa/standalone/mon/health-mute.sh: s/kill daemons/kill_daemons/

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29719 into master
Sage Weil [Mon, 19 Aug 2019 14:09:06 +0000 (09:09 -0500)]
Merge PR #29719 into master

* refs/pull/29719/head:
osd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: E2E Test for RBD Mirroring and Images (#29381)
Lenz Grimmer [Mon, 19 Aug 2019 13:10:59 +0000 (13:10 +0000)]
mgr/dashboard: E2E Test for RBD Mirroring and Images (#29381)

mgr/dashboard: E2E Test for RBD Mirroring and Images

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29685 from smithfarm/wip-41296
Nathan Cutler [Mon, 19 Aug 2019 10:17:46 +0000 (12:17 +0200)]
Merge pull request #29685 from smithfarm/wip-41296

rpm: always build ceph-test package

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
6 years agoMerge pull request #29556 from Devp00l/wip-41165
Lenz Grimmer [Mon, 19 Aug 2019 09:08:43 +0000 (09:08 +0000)]
Merge pull request #29556 from Devp00l/wip-41165

mgr/dashboard: CephFS client tab switch

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: Verify fields on ISCSI page (#29608)
Lenz Grimmer [Mon, 19 Aug 2019 09:03:57 +0000 (09:03 +0000)]
mgr/dashboard: Verify fields on ISCSI page (#29608)

mgr/dashboard: Verify fields on ISCSI page

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: clean-up python unit tests (#28696)
Lenz Grimmer [Mon, 19 Aug 2019 08:52:53 +0000 (08:52 +0000)]
mgr/dashboard: clean-up python unit tests (#28696)

mgr/dashboard: clean-up python unit tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agoMerge pull request #29584 from tchaikov/wip-qa-test_envlibrados_for_rocksdb-cmake3
Kefu Chai [Sun, 18 Aug 2019 04:03:56 +0000 (12:03 +0800)]
Merge pull request #29584 from tchaikov/wip-qa-test_envlibrados_for_rocksdb-cmake3

qa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoqa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake 29584/head
Kefu Chai [Sat, 10 Aug 2019 08:08:00 +0000 (16:08 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake

rocksdb needs CMake 3.5.1 since
https://github.com/facebook/rocksdb/pull/5662

Fixes: https://tracker.ceph.com/issues/41253
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments 29719/head
xie xingguo [Sat, 17 Aug 2019 07:19:46 +0000 (15:19 +0800)]
osd/PrimaryLogPG: kill obsolete ondisk_{read,write}_lock comments

The ondisk_{read,write}_lock infrastructure was long gone with
https://github.com/ceph/ceph/pull/20177 merged - c244300ef33a044ad71fea7d92d77f33b5d41851,
to be specific. Hence the related comments must die since they
could be super-misleading.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #29433 from dillaman/wip-41029
Mykola Golub [Sat, 17 Aug 2019 06:27:02 +0000 (09:27 +0300)]
Merge pull request #29433 from dillaman/wip-41029

pybind/mgr/rbd_support: ignore missing support for RBD namespaces

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #29712 from neha-ojha/wip-basic-workflow
Neha Ojha [Fri, 16 Aug 2019 21:22:06 +0000 (14:22 -0700)]
Merge pull request #29712 from neha-ojha/wip-basic-workflow

doc/dev/developer_guide/index.rst: add youtube reference for Getting Started

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agodoc/dev/developer_guide/index.rst: add youtube ref for Getting Started 29712/head
Neha Ojha [Fri, 16 Aug 2019 19:46:57 +0000 (12:46 -0700)]
doc/dev/developer_guide/index.rst: add youtube ref for Getting Started

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29448 from cbodley/wip-41045
Casey Bodley [Fri, 16 Aug 2019 19:52:11 +0000 (15:52 -0400)]
Merge pull request #29448 from cbodley/wip-41045

rgw: datalog/mdlog trim commands loop until done

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agomgr/dashboard: clean-up tox and unit tests 28696/head
Ernesto Puerta [Fri, 19 Jul 2019 16:10:49 +0000 (18:10 +0200)]
mgr/dashboard: clean-up tox and unit tests

Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.

Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)

Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst

Doc:
- Update HACKING.rst

Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.

Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #29548 into master
Sage Weil [Fri, 16 Aug 2019 16:39:13 +0000 (11:39 -0500)]
Merge PR #29548 into master

* refs/pull/29548/head:
osd: add device_id to list_devices to help get smart info easily

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #29709 from rzarzynski/wip-osd-no-mds-pglsparent
Kefu Chai [Fri, 16 Aug 2019 15:29:07 +0000 (23:29 +0800)]
Merge pull request #29709 from rzarzynski/wip-osd-no-mds-pglsparent

osd: remove orphan include after PGLSParentFilter.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoradosgw-admin: 'mdlog trim' loops until done 29448/head
Casey Bodley [Wed, 31 Jul 2019 21:06:13 +0000 (17:06 -0400)]
radosgw-admin: 'mdlog trim' loops until done

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoradosgw-admin: 'datalog trim' takes shard-id and loops until done
Casey Bodley [Wed, 31 Jul 2019 21:05:40 +0000 (17:05 -0400)]
radosgw-admin: 'datalog trim' takes shard-id and loops until done

Fixes: https://tracker.ceph.com/issues/41045
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29695 from cbodley/wip-41300
Casey Bodley [Fri, 16 Aug 2019 15:21:12 +0000 (11:21 -0400)]
Merge pull request #29695 from cbodley/wip-41300

rgw: RGWSI_User_Module filters '.buckets' objects out of user listing

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoMerge pull request #26787 from soumyakoduri/bucket_name_validation
Casey Bodley [Fri, 16 Aug 2019 14:53:54 +0000 (10:53 -0400)]
Merge pull request #26787 from soumyakoduri/bucket_name_validation

[rgw]:Validate bucket names as per revised s3 spec

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29515 from ivancich/wip-clean-up-reset-stats
Casey Bodley [Fri, 16 Aug 2019 14:21:02 +0000 (10:21 -0400)]
Merge pull request #29515 from ivancich/wip-clean-up-reset-stats

rgw: housekeeping of reset stats operation in radosgw-admin and cls back-end

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #28904 from ZVampirEM77/wip-s3website-fix
Casey Bodley [Fri, 16 Aug 2019 14:20:24 +0000 (10:20 -0400)]
Merge pull request #28904 from ZVampirEM77/wip-s3website-fix

rgw: fix the bug of rgw not doing necessary checking to website configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29215 from tianshan/fix_list_with_delimiter
Casey Bodley [Fri, 16 Aug 2019 14:19:50 +0000 (10:19 -0400)]
Merge pull request #29215 from tianshan/fix_list_with_delimiter

rgw: fix list bucket with delimiter wrongly skip some special keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #29369 from tianshan/fix_realm_watch
Casey Bodley [Fri, 16 Aug 2019 14:18:27 +0000 (10:18 -0400)]
Merge pull request #29369 from tianshan/fix_realm_watch

rgw: fix potential realm watch lost

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29380 from theanalyst/rgw-dns-name-fixes
Casey Bodley [Fri, 16 Aug 2019 14:18:08 +0000 (10:18 -0400)]
Merge pull request #29380 from theanalyst/rgw-dns-name-fixes

rgw: make dns hostnames matching case insensitive

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28558 from cbodley/wip-rgw-bucket-max-aio
Casey Bodley [Fri, 16 Aug 2019 14:17:33 +0000 (10:17 -0400)]
Merge pull request #28558 from cbodley/wip-rgw-bucket-max-aio

[RFC] rgw: raise default rgw_bucket_index_max_aio to 128

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoosd: remove orphan include after PGLSParentFilter. 29709/head
Radoslaw Zarzynski [Fri, 16 Aug 2019 14:15:34 +0000 (16:15 +0200)]
osd: remove orphan include after PGLSParentFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #28824 from dang/zipper
Casey Bodley [Fri, 16 Aug 2019 13:49:28 +0000 (09:49 -0400)]
Merge pull request #28824 from dang/zipper

rgw: Project Zipper part 1

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agomgr/dashboard: clean-up python unit tests
Ernesto Puerta [Fri, 19 Jul 2019 15:23:56 +0000 (17:23 +0200)]
mgr/dashboard: clean-up python unit tests

Fix pylint issues.

Refactor JwtManager check to avoid code duplication (found by pylint
when --jobs=1).

Fix issue with DashboardException.code property, using abs() on
potentially None attribute.

Disables Doctests in services/rbd.py that are actually integration tests
(they check the value of rbd.RBD_FEATURES_NAME_MAPPING). Ideally, these
kind of tests should be explicitly executed in an integration testing
stage, rather that unit-testing. Disabled tests have been prepended with
@DISABLEDOCTEST token.

Fixes: https://tracker.ceph.com/issues/40487
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #29621 from xiexingguo/wip-build-push-segv
Kefu Chai [Fri, 16 Aug 2019 12:25:55 +0000 (20:25 +0800)]
Merge pull request #29621 from xiexingguo/wip-build-push-segv

osd/ReplicatedBackend: check against empty data_included before enabling crc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #29579 from liewegas/wip-big-vs-bluestore
Kefu Chai [Fri, 16 Aug 2019 12:24:43 +0000 (20:24 +0800)]
Merge pull request #29579 from liewegas/wip-big-vs-bluestore

osd: scrub error on big objects; make bluestore refuse to start on big objects

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29663 from cxytt/bugfix
Kefu Chai [Fri, 16 Aug 2019 12:22:32 +0000 (20:22 +0800)]
Merge pull request #29663 from cxytt/bugfix

osd/PrimaryLogPG: Avoid accessing destroyed references in finish_degr…

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29646 from zy751713126/get_pool_name_and_id
Kefu Chai [Fri, 16 Aug 2019 12:21:34 +0000 (20:21 +0800)]
Merge pull request #29646 from zy751713126/get_pool_name_and_id

pybind/rados: add Ioctx.get_pool_id() and Ioctx.get_pool_name()

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29675 from tchaikov/wip-remove-parent-pgls-filter
Kefu Chai [Fri, 16 Aug 2019 12:05:00 +0000 (20:05 +0800)]
Merge pull request #29675 from tchaikov/wip-remove-parent-pgls-filter

osd/PrimaryLogPG: remove unused "parent" pgls-filter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29597 from tchaikov/wip-qa/tasks/cbt
Kefu Chai [Fri, 16 Aug 2019 11:17:58 +0000 (19:17 +0800)]
Merge pull request #29597 from tchaikov/wip-qa/tasks/cbt

qa/tasks/cbt.py: use "git --depth 1 for" faster clone

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29683 from jan--f/c-v-keep-device-list-as-lists
Alfredo Deza [Fri, 16 Aug 2019 11:00:50 +0000 (07:00 -0400)]
Merge pull request #29683 from jan--f/c-v-keep-device-list-as-lists

ceph-volume: don't keep device lists as sets

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PR #29676 into master
Sage Weil [Thu, 15 Aug 2019 19:03:37 +0000 (14:03 -0500)]
Merge PR #29676 into master

* refs/pull/29676/head:
test/unittest_bluefs: always remove temp bdev file

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29581 into master
Sage Weil [Thu, 15 Aug 2019 19:02:34 +0000 (14:02 -0500)]
Merge PR #29581 into master

* refs/pull/29581/head:
os/bluestore: do not set osd_memory_target default from cgroup limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
6 years agoMerge PR #29577 into master
Sage Weil [Thu, 15 Aug 2019 19:02:00 +0000 (14:02 -0500)]
Merge PR #29577 into master

* refs/pull/29577/head:
os/bluestore/KernelDevice: fix RW_IO_MAX constant
os/bluestore/KernelDevice: print aio error extent in hex

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29578 from theanalyst/rgw-user-policy-urlencode
Casey Bodley [Thu, 15 Aug 2019 18:02:03 +0000 (14:02 -0400)]
Merge pull request #29578 from theanalyst/rgw-user-policy-urlencode

rgw: url decode PutUserPolicy params

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoMerge pull request #29670 from linuxbox2/wip-rgwfile-marker
Matt Benjamin [Thu, 15 Aug 2019 17:33:27 +0000 (13:33 -0400)]
Merge pull request #29670 from linuxbox2/wip-rgwfile-marker

rgw_file: readdir: do not construct markers w/leading '/'

6 years agoMerge PR #29422 into master
Sage Weil [Thu, 15 Aug 2019 17:28:26 +0000 (12:28 -0500)]
Merge PR #29422 into master

* refs/pull/29422/head:
qa/tasks/mgr/dashboard/test_health: update schema
doc/rados/operations/monitoring: document muting health alerts
qa/standalone/mon/health-mutes: add tests
doc/rados/operations/health-checks: document MON_DISK_{LOW,CRIT,BIG}
doc/rados/operations/health-checks: document OSD_NO_DOWN_OUT_INTERVAL
doc/rados/operations/health-checks: document AUTH_BAD_CAPS
doc/reados/operations/health-checks: document PG_SLOW_SNAP_TRIMMING
doc/rados/operations/health-checks: document MGR_DOWN
mon/HealthCheck: check mutes based on count, not parsing the summary string
mon/health_checks: associate a count with health_alert_t
mon/HealthMonitor: simplify health alert dump
mon/PGMap: use nice timespan for PG stuck warnings
mon/HealthMonitor: allow muted alert counts to decrease but not increase
mon/PGMap: fix summary form for bluestore health alerts
doc/rados/operations/health-alerts: document BLUESTORE_NO_COMPRESSION
mon/PGMap: fix summary form for POOL_APP_NOT_ENABLED
mon/HealthMonitor: persist summary for non-sticky mutes
mon/HealthMonitor: move get_health_status()
mon/HealthMonitor: automatically clear non-sticky mutes when alert clears
mon/HealthMonitor: add gather_all_health_checks helper
mon/HealthMonitor: add sticky flag to mutes
mon/HealthMonitor: expire mutes based on ttl
mon: apply mutes to health [detail]
mon/HealthMonitor: implement mute and unmount commands
mon/HealthMonitor: maintain list of mutes
mon: refactor/simplify health [detail]
mon/health_checks: format 'health summary' with a colon
mon/health_checks: drop dump_summary_compat

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #29537 into master
Sage Weil [Thu, 15 Aug 2019 17:26:29 +0000 (12:26 -0500)]
Merge PR #29537 into master

* refs/pull/29537/head:
os/bluestore/BlueFS: fix device_migrate_to_* to handle varying alloc sizes
os/bluestore/BlueFS: apply shared_alloc_size to shared device
os/bluestore: whitespace
os/bluestore/BlueFS: add bluefs_shared_alloc_size
os/bluestore/BlueStore.cc: start should be >= _get_ondisk_reserved()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: RGWSI_User_Module filters '.buckets' objects out of user listing 29695/head
Casey Bodley [Thu, 15 Aug 2019 17:13:39 +0000 (13:13 -0400)]
rgw: RGWSI_User_Module filters '.buckets' objects out of user listing

Fixes: https://tracker.ceph.com/issues/41300
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29686 from tchaikov/wip-osdc-wait-for-osdmap
Kefu Chai [Thu, 15 Aug 2019 16:10:25 +0000 (00:10 +0800)]
Merge pull request #29686 from tchaikov/wip-osdc-wait-for-osdmap

osdc: should release the rwlock before waiting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26953 from Exotelis/ceph-dashboard-i18ntool
Lenz Grimmer [Thu, 15 Aug 2019 14:45:51 +0000 (14:45 +0000)]
Merge pull request #26953 from Exotelis/ceph-dashboard-i18ntool

mgr/dashboard: ceph dashboard i18ntool

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29568 from votdev/ignore_dirs
Kefu Chai [Thu, 15 Aug 2019 14:39:05 +0000 (22:39 +0800)]
Merge pull request #29568 from votdev/ignore_dirs

.gitignore: add more stuff

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoceph-volume: don't keep device lists as sets 29683/head
Jan Fajerski [Thu, 15 Aug 2019 10:20:00 +0000 (12:20 +0200)]
ceph-volume: don't keep device lists as sets

This was introduced by #27754. The explicit device lists were cast to
sets but other parts of the code where not updated accordingly. To avoid
touching all code places, only cast to sets for disjoint test and keep
lists otherwise.

Fixes: https://tracker.ceph.com/issues/41292
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #29684 from jan--f/c-v-batch-functional-check-stderr
Jan Fajerski [Thu, 15 Aug 2019 13:19:23 +0000 (15:19 +0200)]
Merge pull request #29684 from jan--f/c-v-batch-functional-check-stderr

ceph-volume: fix batch functional tests, idempotent test must check s…

6 years agomgr/dashboard: Daemons Page Tables Test (#29469)
Lenz Grimmer [Thu, 15 Aug 2019 13:18:33 +0000 (13:18 +0000)]
mgr/dashboard: Daemons Page Tables Test (#29469)

mgr/dashboard: Daemons Page Tables Test

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/dashboard: Logs Page E2E Tests (#29434)
Lenz Grimmer [Thu, 15 Aug 2019 13:17:53 +0000 (13:17 +0000)]
mgr/dashboard: Logs Page E2E Tests (#29434)

mgr/dashboard: Logs Page E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29420 from ricardoasmarques/fix-default-builder-is-not-a-function
Lenz Grimmer [Thu, 15 Aug 2019 13:16:48 +0000 (13:16 +0000)]
Merge pull request #29420 from ricardoasmarques/fix-default-builder-is-not-a-function

mgr/dashboard: Fixes 'defaultBuilder' is not a function

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoProject Zipper Part 1 - Framework and RGWRadosStore 28824/head
Daniel Gryniewicz [Thu, 6 Jun 2019 18:41:55 +0000 (14:41 -0400)]
Project Zipper Part 1 - Framework and RGWRadosStore

This is the first part of Project Zipper, the Store Abstraction Layer.
It introduces the basic framework, and wraps RGWRados in RGWRadosStore.
The goal over the next few weeks is to do the same for user, bucket, and
object.  This will make most of the remaining users of RGWRados wrapped
in SAL classes, allowing it to be completely absorbed into the private
RGWRadosStore.  This will also expose all the APIs that need to be
pusheg up to higher layers in the SAL.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
6 years agoosdc: should release the rwlock before waiting 29686/head
Kefu Chai [Thu, 15 Aug 2019 11:33:15 +0000 (19:33 +0800)]
osdc: should release the rwlock before waiting

this addresses a regresssion introduced by 20b1ac6e

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorpm: always build ceph-test package 29685/head
Nathan Cutler [Thu, 15 Aug 2019 11:28:51 +0000 (13:28 +0200)]
rpm: always build ceph-test package

Fixes: https://tracker.ceph.com/issues/41296
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoceph-volume: fix batch functional tests, idempotent test must check stderr 29684/head
Jan Fajerski [Thu, 15 Aug 2019 11:28:42 +0000 (13:28 +0200)]
ceph-volume: fix batch functional tests, idempotent test must check stderr

Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agoImprove .gitignore 29568/head
Volker Theile [Fri, 9 Aug 2019 07:54:23 +0000 (09:54 +0200)]
Improve .gitignore

Ignore some Python related caching dirs.

Signed-off-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #29536 from batrick/backport-https
Nathan Cutler [Thu, 15 Aug 2019 09:08:01 +0000 (11:08 +0200)]
Merge pull request #29536 from batrick/backport-https

scripts: use https url for redmine

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28997 from b-ranto/wip-push-dash
Boris Ranto [Thu, 15 Aug 2019 09:02:43 +0000 (11:02 +0200)]
Merge pull request #28997 from b-ranto/wip-push-dash

Make ceph-dashboard require grafana dashboards

Reviewed-by: Zack Cerza <zcerza@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agotest/unittest_bluefs: always remove temp bdev file 29676/head
Kefu Chai [Tue, 13 Aug 2019 02:29:24 +0000 (10:29 +0800)]
test/unittest_bluefs: always remove temp bdev file

we leave files in build directory if the test fails. better off
removing them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd, osdc: drop the unused outdata feature of PGLSFilter. 29675/head
Radoslaw Zarzynski [Sat, 10 Aug 2019 08:38:20 +0000 (10:38 +0200)]
osd, osdc: drop the unused outdata feature of PGLSFilter.

Before this commit PGLSFilter interface was offering the outdata
parameter in its filter() method:

  filter(..., bufferlist& outdata)

OSD was serializing and appending the bufferlist to response to
CEPH_OSD_OP_PGLS_FILTER and CEPH_OSD_OP_PGNLS_FILTER operations.
At the Objecter's side these extra bits were being parsed and
finally stored in NListContext::extra_info. However, it really
looks this member is not used anywhere.

The commit removes the outdata handling on multiple layers: from
PGLSFilter implementations, through OSD till Objecter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd/PrimaryLogPG: remove unused "parent" pgls-filter
Kefu Chai [Thu, 15 Aug 2019 02:34:19 +0000 (10:34 +0800)]
osd/PrimaryLogPG: remove unused "parent" pgls-filter

it's implemented using `PGLSParentFilter`, and this filter has never been
used. the only possible user would be `cephfs-data-scan`, but it's using
`PGLSCephFSFilter` which is referened with "cephfs.inode_tag".

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/PrimaryLogPG: Avoid accessing destroyed references in finish_degraded_object 29663/head
Tao Ning [Wed, 14 Aug 2019 13:07:46 +0000 (21:07 +0800)]
osd/PrimaryLogPG: Avoid accessing destroyed references in finish_degraded_object

As follows:
for (auto i = callbacks_for_degraded_object.begin(); i != callbacks_for_degraded_object.end();) {
    finish_degraded_object((i++)->first);
}

void PrimaryLogPG::finish_degraded_object(const hobject_t oid)
{
  if (callbacks_for_degraded_object.count(oid)) {
    contexts.swap(callbacks_for_degraded_object[oid]);
    callbacks_for_degraded_object.erase(oid);   // Release
  }

  map<hobject_t, snapid_t>::iterator i = objects_blocked_on_degraded_snap.find(
    oid.get_head());  // Access
  ...
}

Fixes: https://tracker.ceph.com/issues/41250
Signed-off-by: Tao Ning <ningtao@sangfor.com.cn>
6 years agoos/bluestore/KernelDevice: fix RW_IO_MAX constant 29577/head
Sage Weil [Fri, 9 Aug 2019 14:21:17 +0000 (09:21 -0500)]
os/bluestore/KernelDevice: fix RW_IO_MAX constant

This depends on the page size.  See:

https://github.com/torvalds/linux/blob/6e6d05360b80f196ed07061327f03346b204abea/include/linux/fs.h#L2305

https://github.com/torvalds/linux/blob/30d1d92a888d03681b927c76a35181b4eed7071f/tools/virtio/linux/kernel.h#L23

Fixes 4d33114a40d5ae0d541c36175977ca22789a3b88

Fixes: https://tracker.ceph.com/issues/41188
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/tasks/mgr/dashboard/test_health: update schema 29422/head
Sage Weil [Thu, 8 Aug 2019 16:21:42 +0000 (11:21 -0500)]
qa/tasks/mgr/dashboard/test_health: update schema

Also fix the 'checks' field, which is a list of objects, not strings.  (The
test doesn't notice because it's empty.)

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/monitoring: document muting health alerts
Sage Weil [Thu, 1 Aug 2019 20:00:10 +0000 (15:00 -0500)]
doc/rados/operations/monitoring: document muting health alerts

I think someday the docs for how health alerts work (here) and the
enumeration of all actual alerts should be restructured.  For now this
si the simplest placde to fit this!

Signed-off-by: Sage Weil <sage@redhat.com>t
6 years agoqa/standalone/mon/health-mutes: add tests
Sage Weil [Wed, 31 Jul 2019 22:28:18 +0000 (17:28 -0500)]
qa/standalone/mon/health-mutes: add tests

Make sure mute and unmute work.  Make sure stick is sticky. Mkae sure
counts can go down bupt if they go upt hte mute clears.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/health-checks: document MON_DISK_{LOW,CRIT,BIG}
Sage Weil [Wed, 31 Jul 2019 10:24:20 +0000 (05:24 -0500)]
doc/rados/operations/health-checks: document MON_DISK_{LOW,CRIT,BIG}

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/health-checks: document OSD_NO_DOWN_OUT_INTERVAL
Sage Weil [Wed, 31 Jul 2019 10:15:14 +0000 (05:15 -0500)]
doc/rados/operations/health-checks: document OSD_NO_DOWN_OUT_INTERVAL

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/health-checks: document AUTH_BAD_CAPS
Sage Weil [Wed, 31 Jul 2019 10:10:50 +0000 (05:10 -0500)]
doc/rados/operations/health-checks: document AUTH_BAD_CAPS

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/reados/operations/health-checks: document PG_SLOW_SNAP_TRIMMING
Sage Weil [Wed, 31 Jul 2019 10:04:37 +0000 (05:04 -0500)]
doc/reados/operations/health-checks: document PG_SLOW_SNAP_TRIMMING

The mitigation steps are weak, but it's not clear concrete guidance to
provide.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/health-checks: document MGR_DOWN
Sage Weil [Wed, 31 Jul 2019 09:57:49 +0000 (04:57 -0500)]
doc/rados/operations/health-checks: document MGR_DOWN

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthCheck: check mutes based on count, not parsing the summary string
Sage Weil [Wed, 31 Jul 2019 09:51:38 +0000 (04:51 -0500)]
mon/HealthCheck: check mutes based on count, not parsing the summary string

This is more explicit and robust, and works with the PG warnings, which
don't conform to the "%d ..." form that the other messages do.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/health_checks: associate a count with health_alert_t
Sage Weil [Wed, 31 Jul 2019 09:14:01 +0000 (04:14 -0500)]
mon/health_checks: associate a count with health_alert_t

0 means this is a singleton.  Otherwise, we can sum this up, either
via merge() or get_or_add().

We always structure this so the count goes toward zero (more healthy), so
if a value is too low, then we count how much too low it is.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: simplify health alert dump
Sage Weil [Wed, 31 Jul 2019 08:24:45 +0000 (03:24 -0500)]
mon/HealthMonitor: simplify health alert dump

Use dump() member instead of duplicating!  The only reason we had this
before was because the detail portion was optinoal

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/PGMap: use nice timespan for PG stuck warnings
Sage Weil [Wed, 31 Jul 2019 08:22:38 +0000 (03:22 -0500)]
mon/PGMap: use nice timespan for PG stuck warnings

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: allow muted alert counts to decrease but not increase
Sage Weil [Wed, 31 Jul 2019 07:04:20 +0000 (02:04 -0500)]
mon/HealthMonitor: allow muted alert counts to decrease but not increase

If the summary starts with a digit, parse a count.

If the count goes up, clear the mute.

If the count goes down, update the mute so that we ratchet the threshold
down.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/PGMap: fix summary form for bluestore health alerts
Sage Weil [Wed, 31 Jul 2019 06:39:28 +0000 (01:39 -0500)]
mon/PGMap: fix summary form for bluestore health alerts

Count goes first.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/rados/operations/health-alerts: document BLUESTORE_NO_COMPRESSION
Sage Weil [Wed, 31 Jul 2019 06:39:03 +0000 (01:39 -0500)]
doc/rados/operations/health-alerts: document BLUESTORE_NO_COMPRESSION

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/PGMap: fix summary form for POOL_APP_NOT_ENABLED
Sage Weil [Wed, 31 Jul 2019 06:31:39 +0000 (01:31 -0500)]
mon/PGMap: fix summary form for POOL_APP_NOT_ENABLED

Count goes first.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: persist summary for non-sticky mutes
Sage Weil [Wed, 31 Jul 2019 06:31:22 +0000 (01:31 -0500)]
mon/HealthMonitor: persist summary for non-sticky mutes

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: move get_health_status()
Sage Weil [Wed, 31 Jul 2019 05:27:26 +0000 (00:27 -0500)]
mon/HealthMonitor: move get_health_status()

This operates exclusively on HealthMonitor members.  Make public member
private again.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: automatically clear non-sticky mutes when alert clears
Sage Weil [Wed, 31 Jul 2019 05:19:39 +0000 (00:19 -0500)]
mon/HealthMonitor: automatically clear non-sticky mutes when alert clears

If the alert goes away, drop the mute.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: add gather_all_health_checks helper
Sage Weil [Wed, 31 Jul 2019 05:05:04 +0000 (00:05 -0500)]
mon/HealthMonitor: add gather_all_health_checks helper

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: add sticky flag to mutes
Sage Weil [Wed, 31 Jul 2019 03:54:25 +0000 (22:54 -0500)]
mon/HealthMonitor: add sticky flag to mutes

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: expire mutes based on ttl
Sage Weil [Wed, 31 Jul 2019 00:44:59 +0000 (19:44 -0500)]
mon/HealthMonitor: expire mutes based on ttl

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon: apply mutes to health [detail]
Sage Weil [Wed, 31 Jul 2019 00:44:48 +0000 (19:44 -0500)]
mon: apply mutes to health [detail]

- de-escalate severity
- mark mutes in structured output
- note mutes in summary text output
- mark mutes in detail text output

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: implement mute and unmount commands
Sage Weil [Tue, 30 Jul 2019 16:30:15 +0000 (11:30 -0500)]
mon/HealthMonitor: implement mute and unmount commands

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/HealthMonitor: maintain list of mutes
Sage Weil [Tue, 30 Jul 2019 16:28:55 +0000 (11:28 -0500)]
mon/HealthMonitor: maintain list of mutes

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon: refactor/simplify health [detail]
Sage Weil [Tue, 30 Jul 2019 16:23:57 +0000 (11:23 -0500)]
mon: refactor/simplify health [detail]

Get rid of single caller helpers.  Instead, assimilate all the checks
together at once, and have two separate blocks, one for formatted, and
one for plaintext output.  Much easier to follow!

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/health_checks: format 'health summary' with a colon
Sage Weil [Tue, 30 Jul 2019 15:01:28 +0000 (10:01 -0500)]
mon/health_checks: format 'health summary' with a colon

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/health_checks: drop dump_summary_compat
Sage Weil [Tue, 30 Jul 2019 15:01:15 +0000 (10:01 -0500)]
mon/health_checks: drop dump_summary_compat

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/PG: scrub error when objects are larger than osd_max_object_size 29579/head
David Zafman [Fri, 9 Aug 2019 20:07:22 +0000 (20:07 +0000)]
osd/PG: scrub error when objects are larger than osd_max_object_size

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoMerge PR #29431 into master
Patrick Donnelly [Wed, 14 Aug 2019 22:21:51 +0000 (15:21 -0700)]
Merge PR #29431 into master

* refs/pull/29431/head:
qa: fix malformed suite config

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #28652 into master
Patrick Donnelly [Wed, 14 Aug 2019 22:05:29 +0000 (15:05 -0700)]
Merge PR #28652 into master

* refs/pull/28652/head:
cephfs-shell: Add error message for invalid ls commands

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29554 into master
Patrick Donnelly [Wed, 14 Aug 2019 22:04:04 +0000 (15:04 -0700)]
Merge PR #29554 into master

* refs/pull/29554/head:
cephfs-shell: Fix onecmd TypeError

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29552 into master
Patrick Donnelly [Wed, 14 Aug 2019 22:01:35 +0000 (15:01 -0700)]
Merge PR #29552 into master

* refs/pull/29552/head:
cephfs-shell: Convert paths type from string to bytes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>