]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Mark Nelson [Fri, 12 Jul 2019 16:45:17 +0000 (12:45 -0400)]
common/options: Set concurrent bluestore rocksdb compactions to 2
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit
26dc5189e73a1f5a22f89ee67eef5430ef660b5d )
Andrew Schoen [Mon, 26 Aug 2019 17:21:33 +0000 (12:21 -0500)]
Merge pull request #29900 from alfredodeza/mimic-rm41378
mimic ceph-volume tests set the noninteractive flag for Debian
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Alfredo Deza [Wed, 21 Aug 2019 18:15:32 +0000 (14:15 -0400)]
ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
89231c9a60a46ba0691f787efbfe5ab35f05c679 )
Alfredo Deza [Wed, 21 Aug 2019 12:11:26 +0000 (08:11 -0400)]
Merge pull request #29770 from alfredodeza/mimic-bz-
1738379
mimic ceph-volume: use the OSD identifier when reporting success
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Alfredo Deza [Tue, 20 Aug 2019 12:04:56 +0000 (08:04 -0400)]
ceph-volume devices.lvm zap use the identifier to report success
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
d0d5750c1ba508152c1d5111f2003a039a27145f )
Alfredo Deza [Tue, 20 Aug 2019 12:04:26 +0000 (08:04 -0400)]
ceph-volume tests check success message when zapping
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
88c7429a60fdab6222d14132bea93b9bf773c720 )
Alfredo Deza [Tue, 20 Aug 2019 11:38:45 +0000 (07:38 -0400)]
Merge pull request #29661 from jan--f/wip-41246-mimic
mimic: simple functional tests drop test for lvm zap
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Alfredo Deza [Tue, 20 Aug 2019 11:37:25 +0000 (07:37 -0400)]
Merge pull request #29688 from jan--f/wip-41298-mimic
mimic: batch functional idempotency test fails since message is now on stderr
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Neha Ojha [Fri, 16 Aug 2019 21:07:51 +0000 (14:07 -0700)]
Merge pull request #29618 from pdvian/wip-40943-mimic
mimic: mon/OSDMonitor.cc: better error message about min_size
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 16 Aug 2019 16:31:51 +0000 (09:31 -0700)]
Merge pull request #29699 from pdvian/wip-41086-mimic
mimic: Change default for bluestore_fsck_on_mount_deep as false
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Wed, 31 Jul 2019 20:08:24 +0000 (13:08 -0700)]
common/options.cc: common/options.cc: change default value of bluestore_fsck_on_umount_deep to false
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
8c86c3a5b4bdca258016019f63359f52aa8cd84b )
Neha Ojha [Wed, 31 Jul 2019 00:05:18 +0000 (17:05 -0700)]
common/options.cc: change default value of bluestore_fsck_on_mount_deep to false
Fixes: https://tracker.ceph.com/issues/41017
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
7c03f088f1004d4e26b6bdea8fa58f8455f1d6a2 )
Nathan Cutler [Thu, 15 Aug 2019 15:23:28 +0000 (17:23 +0200)]
Merge pull request #29289 from smithfarm/wip-40497-mimic
mimic: doc: Object Gateway multisite document read-only argument error
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Jan Fajerski [Thu, 15 Aug 2019 11:28:42 +0000 (13:28 +0200)]
ceph-volume: fix batch functional tests, idempotent test must check stderr
Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
88807110f3014e7a2a39c37ae5c64bb3ab107d80 )
Jan Fajerski [Wed, 14 Aug 2019 12:36:58 +0000 (14:36 +0200)]
ceph-volume: don't try to test lvm zap on simple tests
Fixes: https://tracker.ceph.com/issues/41245
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
081109859a06f1b2ff6652dce07da4ddf6fb258e )
Jan Fajerski [Wed, 14 Aug 2019 12:29:51 +0000 (14:29 +0200)]
Merge pull request #29631 from jan--f/backport-mimic-26826
ceph-volume: more mimic backports
Jan Fajerski [Wed, 14 Aug 2019 12:28:33 +0000 (14:28 +0200)]
Merge pull request #29602 from jan--f/wip-41201-mimic
mimic: ceph-volume prints log messages to stdout
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output
Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
f3707f1ad43513c2a6df17e9dce756533c3b4a1d )
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test
This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
b59dcd01c2dd386ca1b63eb66ed243afffb80211 )
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory
Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
1881f386d8207ee8c10f0863c4a1abf18cf3c5d9 )
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests
In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
87c33f149a83bb1175bfb473b226e8c46e4e2a6e )
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
f7943ef6007fb05d83602bc22cbacf3bb67d8625 )
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
7cb098b6899542c875d911d28771a4c450d0bb3a )
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs
If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
28e454ec30f867ae1229c66d263bcd608654553f )
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids
This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
15d20e29a2f2d74caf05dfec559e30941c7a46ab )
Alfredo Deza [Thu, 6 Dec 2018 20:18:06 +0000 (15:18 -0500)]
ceph-volume tests.functional.simple add zap by osd id to xenial
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
f53f67ca3d25deb82c20d31df418a19948ba6f64 )
Alfredo Deza [Thu, 6 Dec 2018 20:17:55 +0000 (15:17 -0500)]
ceph-volume tests.functional.simple add zap by osd id to centos7
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
8955ec14566c48a08bb4826256b422aa29a2c16e )
Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
remove with_items key
Neha Ojha [Tue, 23 Jul 2019 00:50:31 +0000 (17:50 -0700)]
mon/OSDMonitor.cc: better error message about min_size
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
ce25ac0a9196f90acef82915f122c5a06277993a )
Jan Fajerski [Thu, 8 Aug 2019 08:42:37 +0000 (10:42 +0200)]
ceph-volume: never log to stdout, use stderr instead
We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.
Fixes: https://tracker.ceph.com/issues/41158
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
b8d6dcbe9f803c96c0af68da54f1262e9b6a9e77 )
Kefu Chai [Sun, 24 Mar 2019 09:11:31 +0000 (17:11 +0800)]
ceph-volume: terminal: encode unicode when writing to stdout
python determins the encoding of stdout and stderr based on the LC_CTYPE
and PYTHONIOENCODING env variable, by default, python3's sys.stdout uses
'utf-8' as its encoding, so it will be able to write unicode string even
the stdout is not attached to a tty device. but when it comes to
python2, it will default to ascii if neither of these variabls is set.
so, if we are writing unicode using `_Write` in an environment where
LC_CTYPE and/or PYTHONIOENCODING are using non UTF-8 encoding, it chokes
by raising `UnicodeEncodeError` exception.
in this change, we add a wrapper around `_Write._writer` so it is able
to write unicode string in such a non-unicode-friendly environment.
for more info related the encoding of stdout and stderr, see
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING .
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
77912c0c71874a23785d3bdd109ad61e4f4c0e28 )
Jan Fajerski [Wed, 7 Aug 2019 08:18:45 +0000 (10:18 +0200)]
Merge pull request #29507 from jan--f/wip-41138-mimic
mimic: ceph-volume prints errors to stdout with --format json
Jan Fajerski [Mon, 4 Mar 2019 10:28:13 +0000 (11:28 +0100)]
ceph-volume: print most logging messages to stderr
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: http://tracker.ceph.com/issues/38548
(cherry picked from commit
1f8f1e95dee88b4bab2d6931e720ed83584d2887 )
Conflicts:
src/pybind/mgr/test_orchestrator/module.py
removed
Jan Fajerski [Wed, 7 Aug 2019 06:44:00 +0000 (08:44 +0200)]
Merge pull request #29417 from jan--f/wip-41022-mimic
mimic: simple: when 'type' file is not present activate fails
Yuri Weinstein [Tue, 6 Aug 2019 21:19:37 +0000 (14:19 -0700)]
Merge pull request #29255 from smithfarm/wip-39274-mimic
mimic: rgw: S3 policy evaluated incorrectly
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:19:15 +0000 (14:19 -0700)]
Merge pull request #29256 from smithfarm/wip-39415-mimic
mimic: rgw: multisite: period pusher gets 403 Forbidden against other zonegroups
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:18:55 +0000 (14:18 -0700)]
Merge pull request #29262 from smithfarm/wip-39748-mimic
mimic: rgw: Add support for --bypass-gc flag of radosgw-admin bucket rm command in RGW Multi-site
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:18:30 +0000 (14:18 -0700)]
Merge pull request #29267 from smithfarm/wip-40128-mimic
mimic: rgw: Swift interface: server side copy fails if object name contains '?'
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:18:07 +0000 (14:18 -0700)]
Merge pull request #29268 from smithfarm/wip-40136-mimic
mimic: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:17:35 +0000 (14:17 -0700)]
Merge pull request #29270 from smithfarm/wip-40141-mimic
mimic: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:16:18 +0000 (14:16 -0700)]
Merge pull request #29273 from smithfarm/wip-40144-mimic
mimic: rgw: Multisite sync corruption for large multipart obj
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:15:35 +0000 (14:15 -0700)]
Merge pull request #29277 from smithfarm/wip-40225-mimic
mimic: rgw_file: include tenant when hashing bucket names
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:15:06 +0000 (14:15 -0700)]
Merge pull request #29279 from smithfarm/wip-40351-mimic
mimic: rgw/OutputDataSocket: append_output(buffer::list&) says it will (but does not) discard output at data_max_backlog
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:14:44 +0000 (14:14 -0700)]
Merge pull request #29284 from smithfarm/wip-40354-mimic
mimic: rgw: multisite: RGWListBucketIndexesCR for data full sync needs pagination
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:14:21 +0000 (14:14 -0700)]
Merge pull request #29285 from smithfarm/wip-40357-mimic
mimic: rgw: provide admin-friendly reshard status output
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 6 Aug 2019 21:13:54 +0000 (14:13 -0700)]
Merge pull request #29288 from smithfarm/wip-40360-mimic
mimic: rgw: set null version object issues
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Jan Fajerski [Tue, 6 Aug 2019 13:34:12 +0000 (15:34 +0200)]
mimic: ceph-volume broken assertion errors after pytest changes
This fixes a backported test that was not fixed on a backport. Original
fix: https://github.com/ceph/ceph/pull/28948. Backport:
https://github.com/ceph/ceph/pull/29243.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Alfredo Deza [Mon, 5 Aug 2019 20:17:53 +0000 (16:17 -0400)]
Merge pull request #29463 from jan--f/wip-41056-mimic
mimic: ceph-volume does not recognize wal/db partitions created by ceph-disk
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Jan Fajerski [Tue, 30 Jul 2019 15:32:06 +0000 (17:32 +0200)]
ceph-volume: fall back to PARTTYPE if PARTLABEL is empty
In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.
Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
82d2ae7ffdf62ca93c7848fb62e506db4cda893c )
Jan Fajerski [Tue, 30 Jul 2019 15:29:09 +0000 (17:29 +0200)]
ceph-volume: adjust tests for empty PARTLABEL fields
It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
bb29110c3c6e081568231131c70522dee6d12bb5 )
Jan Fajerski [Tue, 30 Jul 2019 11:29:22 +0000 (13:29 +0200)]
ceph-volume: refactor ceph-disk_member unittests
This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
f42287c3416ad09c112e64756f5051c5c13aa344 )
Alfredo Deza [Mon, 29 Jul 2019 19:50:14 +0000 (15:50 -0400)]
ceph-volume tests: ensure that better heuristics exist for objectstore detection
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
e48ad3ab26fdab72755bc933781db3bfc6001a30 )
Alfredo Deza [Mon, 29 Jul 2019 19:46:48 +0000 (15:46 -0400)]
ceph-volume simple.activate better detect bluestore/filestore when type is not found
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b6a449e2e3241aaa2c0eb8fa9f74b5d939c97852 )
Yuri Weinstein [Tue, 30 Jul 2019 16:11:43 +0000 (09:11 -0700)]
Merge pull request #29029 from liewegas/wip-mgr-crash-py3-mimic
mimic: mgr/crash: fix python3 invalid syntax problems
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:11:12 +0000 (09:11 -0700)]
Merge pull request #29150 from pdvian/wip-40269-mimic
mimic: build/ops: make "patch" build dependency explicit
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:10:43 +0000 (09:10 -0700)]
Merge pull request #29197 from smithfarm/wip-38963-mimic
mimic: mgr: DaemonServer::handle_conf_change - broken locking
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:09:58 +0000 (09:09 -0700)]
Merge pull request #29201 from smithfarm/wip-40268-mimic
mimic: common: data race in OutputDataSocket
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 30 Jul 2019 16:09:25 +0000 (09:09 -0700)]
Merge pull request #29217 from smithfarm/wip-39692-mimic
mimic: bluestore: _txc_add_transaction error (39) Directory not empty not handled on operation 21 (op 1, counting from 0)
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Jan Fajerski [Mon, 29 Jul 2019 15:57:41 +0000 (17:57 +0200)]
Merge pull request #29243 from jan--f/mimic-c-v-extend-batch
mimic: c-v extend batch
Andrew Schoen [Tue, 27 Nov 2018 23:34:10 +0000 (17:34 -0600)]
ceph-volume: create two NVME drives for testing
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: bz#
1650306
(cherry picked from commit
893b61b256bcff142dbd90b777b2c9df9e032e9d )
Andrew Schoen [Wed, 28 Nov 2018 16:10:24 +0000 (10:10 -0600)]
ceph-volume: set block_db_size to 9GB in centos7 mixed type test
This is to enable testing of the --block-db-size flag
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
eab962c6ef68c588674bdac6fc4a86c84714f300 )
Andrew Schoen [Wed, 28 Nov 2018 16:05:45 +0000 (10:05 -0600)]
ceph-volume: use two NVME drives in centos7 mixed_type tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
934c32acb318aac9aff3c800cdbf99bbdee04d06 )
Alfredo Deza [Fri, 26 Jul 2019 18:33:21 +0000 (14:33 -0400)]
Merge pull request #29350 from jan--f/wip-40977-mimic
mimic: ceph-volume: missing string substitution when reporting mounts
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Shyukri Shyukriev [Fri, 7 Jun 2019 12:09:19 +0000 (12:09 +0000)]
ceph-volume:util: Use proper param substition
Fixes: https://tracker.ceph.com/issues/25030
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit
654660cd89fe9d93087d390805651c860bf4e862 )
Alfredo Deza [Thu, 25 Jul 2019 15:54:37 +0000 (11:54 -0400)]
Merge pull request #26991 from ceph/backport-mimic-26957
mimic: ceph-volume: look for rotational data in lsblk
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Andrew Schoen [Thu, 14 Mar 2019 20:14:40 +0000 (15:14 -0500)]
ceph-volume: look for rotational data in lsblk
Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=
1666822
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
de748f9aef894b247622ee5dc49c1fbc31097ad0 )
Yuri Weinstein [Thu, 25 Jul 2019 14:37:25 +0000 (07:37 -0700)]
Merge pull request #28918 from smithfarm/wip-40222-mimic
mimic: mds: reset heartbeat during long-running loops in recovery
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:36:55 +0000 (07:36 -0700)]
Merge pull request #29187 from xiaoxichen/wip-40875-mimic
mimic: cephfs: client: bump ll_ref from int32 to uint64_t
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:36:25 +0000 (07:36 -0700)]
Merge pull request #29200 from smithfarm/wip-39685-mimic
mimic: cephfs: ceph-fuse: client hang because its bad session PipeConnection to mds
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:35:41 +0000 (07:35 -0700)]
Merge pull request #29208 from smithfarm/wip-40317-mimic
mimic: tests: test_volume_client: print python version correctly
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:35:08 +0000 (07:35 -0700)]
Merge pull request #29210 from smithfarm/wip-38099-mimic
mimic: mds: remove cache drop admin socket command
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Jul 2019 14:34:17 +0000 (07:34 -0700)]
Merge pull request #29211 from smithfarm/wip-38687-mimic
mimic: kcephfs TestClientLimits.test_client_pin fails with client caps fell below min
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:06:01 +0000 (16:06 -0700)]
Merge pull request #27028 from mrandybu/mimic-alt
mimic: ceph_detect_init: Add support for ALT Linux
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:04:23 +0000 (16:04 -0700)]
Merge pull request #28210 from votdev/fix_rgw_issue
mimic: mgr/dashboard: Fix various RGW issues
Reviewed-by: Tiago Melo <tmelo@suse.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:03:24 +0000 (16:03 -0700)]
Merge pull request #28911 from bk201/wip-40660-mimic
mimic: mgr/dashboard: cephfs multimds graphs stack together
Reviewed-by: Ricardo Dias <rdias@suse.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:02:19 +0000 (16:02 -0700)]
Merge pull request #29024 from ifed01/wip-ifed-fix-len0-mimic
mimic: os/bluestore: avoid length overflow in extents returned by Stupid Allocator
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 23:00:55 +0000 (16:00 -0700)]
Merge pull request #29174 from neha-ojha/wip-40583-mimic
mimic: common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:59:50 +0000 (15:59 -0700)]
Merge pull request #29189 from badone/wip-mimic-lazy-omap-stats-backport-tracker-38552
mimic: core: lazy omap stat collection
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 22:58:23 +0000 (15:58 -0700)]
Merge pull request #29196 from smithfarm/wip-37749-mimic
mimic: tools: Add clear-data-digest command to objectstore tool
Reviewed-by: David Zafman <dzafman@redhat.com>
Casey Bodley [Mon, 3 Jun 2019 14:59:00 +0000 (10:59 -0400)]
test/rgw: test truncated reads in RGWHTTPManager
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
c669cf293f02b3c68a4c539d7accf7ddf35cb1ee )
Conflicts:
src/test/rgw/test_http_manager.cc
- RGWHTTP::process() only takes one argument in mimic
Chenjiong Deng [Wed, 19 Jun 2019 10:17:43 +0000 (18:17 +0800)]
doc: fixed --read-only argument value in multisite doc
Fixes: https://tracker.ceph.com/issues/40458
Signed-off-by: Chenjiong Deng <dengchenjiong@umcloud.com>
(cherry picked from commit
c65832c1ff03e802f98cb7f4bda620e1e515264f )
Tianshan Qu [Sun, 11 Nov 2018 11:56:51 +0000 (19:56 +0800)]
rgw: set null version object acl issues
1.set null version object acl will create empty index
RGWRados::set_attrs did not clear instance, so index prepare, complete got instance=null,
which lead to empty index 1000_<obj>_i_null.
there is no harm to create empty index, but listomapkeys to find that key.
2.if object is exist with versioned key, we can set none exists null version object
order:
1) enable bucket version
2) put obj
3) disable bucket version
4) set versoned_id=null acl will succeed which should not
Fixes: http://tracker.ceph.com/issues/36763
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit
ca6e5c89e8d6da498da15f0eea4bcd4d271b44e9 )
Mark Kogan [Sun, 12 May 2019 13:45:30 +0000 (09:45 -0400)]
rgw: provide admin friendly reshard status output
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit
ba4167c2f30fbb868482d48aaada6388f58de920 )
Mark Kogan [Tue, 11 Dec 2018 17:43:32 +0000 (12:43 -0500)]
rgw: update the "radosgw-admin reshard status"
command documentation with expected output examples
Fixes: http://tracker.ceph.com/issues/37615
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit
8565ec0db69223ae71766ba000eecc0d2f7d14c2 )
Shilpa Jagannath [Thu, 16 May 2019 09:15:46 +0000 (14:45 +0530)]
rgw/multisite:RGWListBucketIndexesCR for data full sync pagination
RGWListBucketIndexesCR uses the 'marker' from remote zone's RGWOp_Metadata_List
admin api to list all keys for data sync.
Fixes http://tracker.ceph.com/issues/39551
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit
708ba6dcc90525c8c4316760e01ee9b1e6da21b7 )
Conflicts:
src/rgw/rgw_data_sync.cc
- trivial
Yuri Weinstein [Wed, 24 Jul 2019 15:26:33 +0000 (08:26 -0700)]
Merge pull request #29016 from ivancich/mimic-abort-multipart-fix
mimic: rgw: abort multipart fix
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:26:07 +0000 (08:26 -0700)]
Merge pull request #29021 from smithfarm/wip-39074-mimic
mimic: rgw: multisite: data sync loops back to the start of the datalog after reaching the end
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:25:38 +0000 (08:25 -0700)]
Merge pull request #29139 from ivancich/mimic-fix-resharding-race
mimic: rgw: fix race b/w bucket reshard and ops waiting on reshard completion
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 24 Jul 2019 15:22:59 +0000 (08:22 -0700)]
Merge pull request #29017 from smithfarm/wip-38080-mimic
mimic: multisite: overwrites in versioning-suspended buckets fail to sync
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Matt Benjamin [Wed, 5 Jun 2019 17:25:32 +0000 (13:25 -0400)]
rgw/OutputDataSocket: actually discard data on full buffer
A dout message in OutputDataSocket::append_output() states that
data will be dropped when appending would cause data_max_backlog
to be exceeded--but the method appends it anyway.
Log output discards at level 0, as messages will be lost. Suppress
repeated warnings mod 100. Switch to vector.
Fixes: http://tracker.ceph.com/issues/40178
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
c806b825dae649829de8847d36cb21ffd2bbee8e )
Conflicts:
src/common/OutputDataSocket.cc
src/common/OutputDataSocket.h
Matt Benjamin [Wed, 5 Jun 2019 14:40:08 +0000 (10:40 -0400)]
rgw_file: pretty-print fh_key
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
84a2c22108a3d15b836680894ba750b637e49cd2 )
Matt Benjamin [Wed, 5 Jun 2019 11:25:35 +0000 (07:25 -0400)]
rgw_file: include tenant in hashes of object
Because bucket names are taken as object names in the top
of an export. Make hashing by tenant general to avoid disjoint
hashing of bucket.
Fixes: http://tracker.ceph.com/issues/40118
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
8e0fd5fbfa7c770f6b668e79b772179946027bce )
Matt Benjamin [Wed, 15 May 2019 19:53:32 +0000 (15:53 -0400)]
rgw_file: include tenant when hashing bucket names
Prevent identical paths from distinct tenants from colliding in
RGW NFS handle cache.
Fixes: http://tracker.ceph.com/issues/40118
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
b800a9de83dff23a150ed7d236cb61c8b7d971ae )
Conflicts:
src/rgw/rgw_file.cc
- ceph_assert
Tianshan Qu [Fri, 31 May 2019 10:37:44 +0000 (18:37 +0800)]
rgw: fix miss handle curl error return
if meet tcp packet loss, curl return will be result=18(CURLE_PARTIAL_FILE)
and http_status=200, so sync will continue and cause content miss match.
Fixes: https://tracker.ceph.com/issues/39992
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit
9b7025857026c64cf076e82d762556ac2bcd2625 )
Jan Fajerski [Wed, 16 Jan 2019 12:48:25 +0000 (13:48 +0100)]
ceph-volume: add explicit mixed-type functional tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
bea35c63c83fc2ab87fc983774ed8181cbe74e5b )
Casey Bodley [Wed, 29 May 2019 21:11:13 +0000 (17:11 -0400)]
rgw: fetch_remote_obj() compares expected object size
if the size of object data received doesn't match the value returned in
the Rgwx-Object-Size header, fail with -EIO
Fixes: https://tracker.ceph.com/issues/39992
Signed-off-by: Xiaoxi CHEN <xiaoxchen@ebay.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
c6c4c5639ea34c49329719c2725886e740acd0b3 )
Casey Bodley [Fri, 24 May 2019 20:59:34 +0000 (16:59 -0400)]
radosgw-admin: fix syncs_from in 'bucket sync status'
If a zone is set up with sync_from_all=false, the bucket sync status
command will incorrectly report "not in sync_from" because it's calling
'zone.syncs_from(source.id)', where syncs_from() expects a zone name
instead of id
Fixes: http://tracker.ceph.com/issues/40022
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
9da888686c3a8880976cd604bdafc4361068c985 )
Conflicts:
src/rgw/rgw_zone.h
- in mimic, this code is in src/rgw/rgw_rados.h - made the change there instead
Radoslaw Zarzynski [Tue, 13 Dec 2016 16:19:56 +0000 (17:19 +0100)]
rgw: dump s3_code as the Code response element in RGWDeleteMultiObj_ObjStore_S3.
Fixes: http://tracker.ceph.com/issues/18241
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit
8406083fb14042cb461a3aaa84b82fd78df5168f )
Andrew Schoen [Thu, 6 Dec 2018 18:45:10 +0000 (12:45 -0600)]
ceph-volume: add tests for Device.lvm_size
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
01185fb32479f271beeb14dcf94e32b1401ad2d9 )
Andrew Schoen [Wed, 5 Dec 2018 15:43:28 +0000 (09:43 -0600)]
ceph-volume: adds a lvm_size property to util.Device
This will give us the size lvm should report because it takes into
account the 1GB physical extent size we set.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
5197c16b44fcf344862201549777dc2935fc19a3 )
Casey Bodley [Tue, 11 Dec 2018 21:13:58 +0000 (16:13 -0500)]
rgw: parse_copy_location defers url-decode
don't url-decode until after we search for the ?, or we'll truncate
object names that contain a url-encoded ?
Fixes: http://tracker.ceph.com/issues/27217
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
804bb853f1c7b5ce546c100d61c6fc526838c134 )