]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoMerge pull request #20672 from linuxbox2/luminous-21942
Yuri Weinstein [Tue, 20 Mar 2018 21:09:14 +0000 (14:09 -0700)]
Merge pull request #20672 from linuxbox2/luminous-21942

luminous: rgw_log, rgw_file: account for new required envvars

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
7 years agoMerge pull request #20741 from pdvian/wip-23179-luminous
Yuri Weinstein [Tue, 20 Mar 2018 21:05:56 +0000 (14:05 -0700)]
Merge pull request #20741 from pdvian/wip-23179-luminous

luminous: rgw: can't download object with range when compression enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20157 from pdvian/wip-22812-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:13:31 +0000 (07:13 -0700)]
Merge pull request #20157 from pdvian/wip-22812-luminous

luminous: rgw: log the right http status code in civetweb frontend's access log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20252 from pdvian/wip-22858-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:13:05 +0000 (07:13 -0700)]
Merge pull request #20252 from pdvian/wip-22858-luminous

luminous: rgw: allow beast frontend to listen on specific IP address

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20345 from pdvian/wip-22889-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:12:32 +0000 (07:12 -0700)]
Merge pull request #20345 from pdvian/wip-22889-luminous

luminous: rgw_file: avoid evaluating nullptr for readdir offset

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
7 years agoMerge pull request #20476 from pdvian/wip-23020-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:11:46 +0000 (07:11 -0700)]
Merge pull request #20476 from pdvian/wip-23020-luminous

luminous: rgw: fix the max-uploads parameter not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20477 from pdvian/wip-23022-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:11:17 +0000 (07:11 -0700)]
Merge pull request #20477 from pdvian/wip-23022-luminous

luminous: rgw: upldate the max-buckets when the quota is uploaded

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20586 from pdvian/wip-23102-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:10:27 +0000 (07:10 -0700)]
Merge pull request #20586 from pdvian/wip-23102-luminous

luminous: rgw: parse old rgw_obj with namespace correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20072 from smithfarm/wip-22766-luminous
Yuri Weinstein [Tue, 20 Mar 2018 14:03:46 +0000 (07:03 -0700)]
Merge pull request #20072 from smithfarm/wip-22766-luminous

luminous: RGW doesn't check time skew in auth v4 http header request

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20906 from pdvian/wip-23317-luminous
Abhishek L [Mon, 19 Mar 2018 15:14:55 +0000 (16:14 +0100)]
Merge pull request #20906 from pdvian/wip-23317-luminous

luminous: doc: rgw: mention the civetweb support for binding to multiple ports

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoMerge pull request #20242 from cbodley/wip-doc-rgw-frontends-luminous
Abhishek L [Mon, 19 Mar 2018 14:40:57 +0000 (15:40 +0100)]
Merge pull request #20242 from cbodley/wip-doc-rgw-frontends-luminous

luminous: doc/rgw: add page for http frontend configuration

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoMerge pull request #20923 from ceph/backport-20910
Alfredo Deza [Fri, 16 Mar 2018 12:43:13 +0000 (08:43 -0400)]
Merge pull request #20923 from ceph/backport-20910

luminous: ceph-volume: fix action plugins path in tox

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: fix action plugins path in tox 20923/head
Guillaume Abrioux [Thu, 15 Mar 2018 08:45:28 +0000 (09:45 +0100)]
ceph-volume: fix action plugins path in tox

because of this commit [1] in ceph-ansible tests in ceph-volume needs to
be modified accordingly.

[1] https://github.com/ceph/ceph-ansible/commit/60d4b75f519c03fca91384f231b071793f582376

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 9fdc56717e72e79dca49fc22da37fd185369f216)

7 years agodoc: rgw: mention the civetweb support for binding to multiple ports 20906/head
Abhishek Lekshmanan [Tue, 22 Aug 2017 12:03:01 +0000 (14:03 +0200)]
doc: rgw: mention the civetweb support for binding to multiple ports

We support binding multiple ports in civetweb after the upgrade to 1.8,
clarify this in the documentation as well

Fixes: http://tracker.ceph.com/issues/20942
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 01740c6a6c1ae523ac73c24235be0e4cef71e21c)

7 years agoMerge pull request #20851 from jdurgin/wip-pg-log-trim-error-luminous 20893/head
Josh Durgin [Mon, 12 Mar 2018 21:12:10 +0000 (14:12 -0700)]
Merge pull request #20851 from jdurgin/wip-pg-log-trim-error-luminous

osd: treat successful and erroroneous writes the same for log trimming

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoPrimaryLogPG: only trim up to osd_pg_log_trim_max entries at once 20851/head
Josh Durgin [Thu, 8 Mar 2018 03:31:59 +0000 (22:31 -0500)]
PrimaryLogPG: only trim up to osd_pg_log_trim_max entries at once

This prevents the fix for http://tracker.ceph.com/issues/22050 or
potential future bugs from causing too much latency by trimming too
many log entries at once.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 1c15458a00e64d1fd0127ae171305b8f8afa6619)

7 years agoPG, PrimaryLogPG: trim log and rollback info for error log entries
Josh Durgin [Thu, 8 Mar 2018 03:09:19 +0000 (22:09 -0500)]
PG, PrimaryLogPG: trim log and rollback info for error log entries

Regular updates piggyback some osd state for this purpose with
MOSDRepOp[Reply]. Do the same thing for pure log entry updates (write
errors and lost/revert additions) via MOSDPGUpdateLogMissing[Reply].

Fixes: http://tracker.ceph.com/issues/22050
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit b50186bfe6c8981700e33c8a62850e21779d67d5)

Conflicts:
qa/standalone/osd/repro_long_log.sh
src/messages/MOSDPGUpdateLogMissing.h
src/messages/MOSDPGUpdateLogMissingReply.h
trival, encode/decode not namespaced

7 years agoMerge pull request #20817 from tchaikov/wip-yet-another-23281-luminous
Nathan Cutler [Fri, 9 Mar 2018 12:53:27 +0000 (13:53 +0100)]
Merge pull request #20817 from tchaikov/wip-yet-another-23281-luminous

test/ceph-disk: specify the python used for creating venv

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agotest/ceph-disk: specify the python used for creating venv 20817/head
Kefu Chai [Fri, 9 Mar 2018 07:27:05 +0000 (15:27 +0800)]
test/ceph-disk: specify the python used for creating venv

run-tox-ceph-disk always fails with following error

py27 runtests: commands[0] | coverage run --append --source=ceph_disk
/home/jenkins-build/build/workspace/ceph-pull-requests/src/ceph-disk/.tox/py27/bin/py.test
-vv
/home/jenkins-build/build/workspace/ceph-pull-requests/src/ceph-disk/tests/test_main.py
ERROR: InvocationError: could not find executable 'coverage'

but the log shows that coverage==3.7.1 was indeed installed in the py27
env. i tried to specify the full path of coverage in "commands" section
in tox.ini, but the problem persisted -- this time the backtrace was
printed, and the raised exception was 'OSError: [Errno 2] No such file
or directory'. so try to use the specified python for creating venv.

Fixes: http://tracker.ceph.com/issues/23281
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
        src/ceph-disk/tox.ini: the issue described above only exists
in luminous branch, so this change is not cherry-picked from master.

7 years agoMerge pull request #20803 from dzafman/wip-zafman-log-trim-luminous
David Zafman [Fri, 9 Mar 2018 00:43:24 +0000 (16:43 -0800)]
Merge pull request #20803 from dzafman/wip-zafman-log-trim-luminous

luminous: ceph-objectstore-tool command to trim the pg log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agotools: Add pg log trim command to ceph-objectstore-tool 20803/head
Josh Durgin [Tue, 6 Mar 2018 01:04:10 +0000 (20:04 -0500)]
tools: Add pg log trim command to ceph-objectstore-tool

Add test script that verifies the command in qa/standalone/osd

Fixes: http://tracker.ceph.com/issues/23242
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b01e4ea5e2c1d2e9e997f53621aba73a7f7c9241)

Conflicts:
src/tools/ceph_objectstore_tool.cc
Different ops
Need ObjectStore::Sequencer
Use apply_transaction()
qa/standalone/special/ceph_objectstore_tool.py
Different ops

7 years agoMerge pull request #20754 from ceph/backport-rm23163
Alfredo Deza [Wed, 7 Mar 2018 12:45:52 +0000 (07:45 -0500)]
Merge pull request #20754 from ceph/backport-rm23163

luminous: ceph-volume tests alleviate libvirt timeouts when reloading

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agorgw: fix return value of auth v2/v4 20072/head
Bingyin Zhang [Thu, 14 Dec 2017 08:38:35 +0000 (16:38 +0800)]
rgw: fix return value of auth v2/v4

* The return value of auth v2/v4 in RGW is different from that in AWS:
*     1. When 'Expires' is missing in auth v2 query string request, AWS
*     returns AccessDenied while RGW returns SignatureDoesNotMatch;
*     2. When 'X-Amz-Expires' is missing in auth v4 query string
*     request, AWS returns AuthorizationQueryParametersError while RGW
*     returns RequestTimeTooSkewed;
* Changes:
*     1. When 'Expires' is missing in auth v2 query string request,
*     change RGW's return value to AccessDenied;
*     2. When 'X-Amz-Expires' is missing in auth v4 query string
*     request, change RGW's return value to AccessDenied;
*     3. remove time skew check from parse_v4_query_string;

Fixes: http://tracker.ceph.com/issues/22439
Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>
(cherry picked from commit ce42f1e8f51b71b242c17077d01fc3009d370e78)

7 years agorgw: share time skew check between v2 and v4 auth
Casey Bodley [Tue, 9 Jan 2018 21:12:38 +0000 (16:12 -0500)]
rgw: share time skew check between v2 and v4 auth

this moves the new std::chrono-based v4 time skew check into a common
helper function, then uses that in place of the v2 check

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit bc98772bddab44bf60fd2b081bff58198b7c043e)

Conflicts:
     src/rgw/rgw_auth_s3.cc - avoid C++17-ism (std::chrono::abs) in
                              is_time_skew_ok()

7 years agorgw: optimize time skew check
Bingyin Zhang [Thu, 14 Dec 2017 08:03:36 +0000 (16:03 +0800)]
rgw: optimize time skew check

* Move query string check out of function is_time_skew_ok;

Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>
(cherry picked from commit 2c97b0ce8378dedef9296bd57273c0b9f1c1f440)

7 years agorgw: add time skew check in function parse_v4_auth_header
Bingyin Zhang [Wed, 13 Dec 2017 10:08:53 +0000 (18:08 +0800)]
rgw: add time skew check in function parse_v4_auth_header

* In auth v4 http header request, RGW doesn't check time skew, while AWS
* does.

Fixes: http://tracker.ceph.com/issues/22418
Signed-off-by: Bingyin Zhang <zhangbingyin@cloudin.cn>
(cherry picked from commit b4e2f32206869501b808c97bcbfbbbef82eb8cf4)

7 years agoceph-volume tests alleviate libvirt timeouts when reloading 20753/head 20754/head
Alfredo Deza [Mon, 5 Mar 2018 12:58:42 +0000 (07:58 -0500)]
ceph-volume tests alleviate libvirt timeouts when reloading

See vagrant-libvirt comment:

    https://github.com/vagrant-libvirt/vagrant-libvirt/issues/510#issuecomment-195326272

And ceph-ansible commit:

    https://github.com/ceph/ceph-ansible/commit/eae6ad6d40561f70679b4973d1bd4b39febd941c

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 14077475685fe1c37ced9a9e4194183323a46760)

7 years agoMerge pull request #20716 from ceph/backport-rm22841
Andrew Schoen [Tue, 6 Mar 2018 13:09:23 +0000 (14:09 +0100)]
Merge pull request #20716 from ceph/backport-rm22841

luminous ceph-volume be resilient to $PATH issues

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agorgw: download object might fail for local invariable uninitialized 20741/head
fang yuxiang [Tue, 27 Feb 2018 17:05:22 +0000 (01:05 +0800)]
rgw: download object might fail for local invariable uninitialized

This is an omission in #20226

Fixes: http://tracker.ceph.com/issues/23146
Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>
(cherry picked from commit f5d2a666ed26574e2e4ec54470939e4acf29796a)

7 years agorgw: can't download object with range when compression enabled
fang yuxiang [Thu, 1 Feb 2018 09:36:46 +0000 (17:36 +0800)]
rgw: can't download object with range when compression enabled

Fixes: http://tracker.ceph.com/issues/22852
Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>
(cherry picked from commit 1e3af175359d633c3728524396a22f9f33873ff2)

7 years agoceph-volume process always try to expand the executable 20716/head
Alfredo Deza [Wed, 28 Feb 2018 16:59:35 +0000 (11:59 -0500)]
ceph-volume process always try to expand the executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 161f331382f8efd0b58ffc25a54187518548abf2)

7 years agoceph-volume tests.util ensure behavior of new which utility
Alfredo Deza [Mon, 26 Feb 2018 21:27:38 +0000 (16:27 -0500)]
ceph-volume tests.util ensure behavior of new which utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d26215a49b5d850a2bff6e75cfef7a036f1da60d)

7 years agoceph-volume util.system add a helper to find system executables
Alfredo Deza [Mon, 26 Feb 2018 21:17:12 +0000 (16:17 -0500)]
ceph-volume util.system add a helper to find system executables

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c36afed9a2fd956dd0694f247f931ae92ae6629a)

7 years agoMerge pull request #20691 from ceph/backport-wip-rm23148
Alfredo Deza [Sun, 4 Mar 2018 14:17:06 +0000 (09:17 -0500)]
Merge pull request #20691 from ceph/backport-wip-rm23148

luminous docs fix ceph-volume missing sub-commands

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #20679 from ceph/backport-rm22663
Andrew Schoen [Fri, 2 Mar 2018 15:25:29 +0000 (16:25 +0100)]
Merge pull request #20679 from ceph/backport-rm22663

luminous doc: update man page to explain ceph-volume support bluestore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agodoc/rados add missing lvm subcommand for ceph-volume 20691/head
Alfredo Deza [Thu, 1 Mar 2018 19:21:05 +0000 (14:21 -0500)]
doc/rados add missing lvm subcommand for ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1c846f3389db46f35adca57861665cd0aaab9b7a)

7 years agodoc/rados update bluestore-migration to use ceph-volume
Alfredo Deza [Wed, 29 Nov 2017 14:57:29 +0000 (09:57 -0500)]
doc/rados update bluestore-migration to use ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 814b5b1658b0de4a58c1ecbfc0687a99b4513fd3)

7 years agodoc: correcting typos in bluestore-config-ref and bluestore-migration
Katie Holly [Sun, 26 Nov 2017 16:47:35 +0000 (17:47 +0100)]
doc: correcting typos in bluestore-config-ref and bluestore-migration

Signed-off-by: Katie Holly <git@meo.ws>
(cherry picked from commit 40e20986cddddb6bea13905a66a8b1fa554af21f)

7 years agodoc: fix typo in bluestore-migration.rst
Yao Zongyou [Thu, 19 Oct 2017 06:38:40 +0000 (14:38 +0800)]
doc: fix typo in bluestore-migration.rst

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit ae660d902b9a57333aac87764e889c736597e557)

7 years agodoc: restructure bluestore migration insructions
Sage Weil [Fri, 8 Sep 2017 15:13:49 +0000 (11:13 -0400)]
doc: restructure bluestore migration insructions

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9fa490178341db50188fb61501cb5f71223ce7c1)

7 years agodoc/rados/operations/bluestore-migration: Add instruction for evacuating
David Galloway [Fri, 8 Sep 2017 14:46:21 +0000 (10:46 -0400)]
doc/rados/operations/bluestore-migration: Add instruction for evacuating
host

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit a9314f0a172b9607180a9212fbe405654365d56a)

7 years agodoc/rados/operations/bluestore-migration: Fix typo in bluestore migration doc
David Galloway [Thu, 7 Sep 2017 23:51:45 +0000 (19:51 -0400)]
doc/rados/operations/bluestore-migration: Fix typo in bluestore migration doc

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit f1b3d1e4bbffb1d8da99951ddfa13b2d1cb6a19d)

7 years agodoc/rados/operations/bluestore-migration: use the new 'safe-to-destroy' command
Sage Weil [Sun, 13 Aug 2017 19:07:07 +0000 (15:07 -0400)]
doc/rados/operations/bluestore-migration: use the new 'safe-to-destroy' command

Also use 'ceph osd ls-tree' to enumerate the ids.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 39c02728b7db3537621167f29651aef97184e8e4)

7 years agodoc/rados/operations/bluestore-migration: fix tree output
Sage Weil [Sun, 13 Aug 2017 19:06:46 +0000 (15:06 -0400)]
doc/rados/operations/bluestore-migration: fix tree output

I forgot to update the osd ids.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9086e1df78ec4eb820eb7448f7a03e0fe6da3781)

7 years agodoc/rados/operations/bluestore-migration: typos and whitespace
Sage Weil [Tue, 8 Aug 2017 19:09:50 +0000 (15:09 -0400)]
doc/rados/operations/bluestore-migration: typos and whitespace

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a1512b023948a25874a4c5b09b32631287d1a9d)

7 years agodoc/rados/operations/bluestore-migration: document bluestore migration process
Sage Weil [Tue, 8 Aug 2017 19:09:50 +0000 (15:09 -0400)]
doc/rados/operations/bluestore-migration: document bluestore migration process

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f7398ddd23321be85e791ea1663c8052632c54f2)

7 years agodoc/install add missing lvm subcommand for ceph-volume
Alfredo Deza [Thu, 1 Mar 2018 19:22:01 +0000 (14:22 -0500)]
doc/install add missing lvm subcommand for ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c66300db8342f6f854bbb7ec8f45a9f08bc5a99c)

7 years agodoc/install use min_size 2 and default_size 4 as an example
Alfredo Deza [Fri, 22 Sep 2017 12:51:53 +0000 (08:51 -0400)]
doc/install use min_size 2 and default_size 4 as an example

Which prevents readin "min_size=1" and assuming it is an OK setting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3c4e4af35194e2b1e12ae747a3bf8e3ab0dc7c4)

7 years agodoc/install use ceph-volume in manual deployment steps
Alfredo Deza [Wed, 29 Nov 2017 16:13:47 +0000 (11:13 -0500)]
doc/install use ceph-volume in manual deployment steps

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3ed739e541ec45ce21845768bc043283315cf232)

7 years agodoc: update man page to explain ceph-volume support bluestore 20679/head
lijing [Wed, 17 Jan 2018 02:18:59 +0000 (10:18 +0800)]
doc: update man page to explain ceph-volume support bluestore

Fixes: http://tracker.ceph.com/issues/22663
Signed-off-by: Jing Li lijing@gohighsec.com
(cherry picked from commit c8bdb49c927bc68d88c6de12537bd1daf7e2e433)

7 years agoceph-volume: update argument description that bluestore is the default
lijing [Wed, 17 Jan 2018 02:20:11 +0000 (10:20 +0800)]
ceph-volume: update argument description that bluestore is the default

Fixes: http://tracker.ceph.com/issues/22663
Signed-off-by: Jing Li lijing@gohighsec.com
(cherry picked from commit 7a0c4d02df0efdbe689b9c89f014b9692b76b79e)

7 years agorgw_log, rgw_file: account for new required envvars 20672/head
Matt Benjamin [Fri, 27 Oct 2017 00:31:34 +0000 (20:31 -0400)]
rgw_log, rgw_file: account for new required envvars

rgw_log_op() expects more server and request variables from the
request environment.  Fix rgw_lib to set more that it can
usefully, but also make rgw_log_op() check for the existence
of all keys it will use.

Fixes: http://tracker.ceph.com/issues/21942
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit f9d5528a1990b5bcc33948284136c2d349aa838e)

7 years agoMerge pull request #20647 from ceph/backport-20620
Alfredo Deza [Wed, 28 Feb 2018 19:59:51 +0000 (14:59 -0500)]
Merge pull request #20647 from ceph/backport-20620

luminous: ceph-volume: adds raw device support to 'lvm list'

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20641 from smithfarm/wip-23159-luminous
Sage Weil [Wed, 28 Feb 2018 14:31:11 +0000 (08:31 -0600)]
Merge pull request #20641 from smithfarm/wip-23159-luminous

luminous: tests: drop upgrade/jewel-x/point-to-point-x in luminous and master

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoceph-volume: if using lvm list on a device with many lvs, show them all 20647/head
Andrew Schoen [Tue, 27 Feb 2018 15:02:12 +0000 (09:02 -0600)]
ceph-volume: if using lvm list on a device with many lvs, show them all

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 729ed7c98ec9833b3826d42d91d9a7e4cbf876b2)

7 years agoceph-volume: lvm list now works with raw devices
Andrew Schoen [Mon, 26 Feb 2018 20:43:40 +0000 (14:43 -0600)]
ceph-volume: lvm list now works with raw devices

If you used a raw device with --data to create an OSD
you can now use 'lvm list' to get information about that OSD
using the raw device name like:

ceph-volume lvm list /dev/sda

fixes http://tracker.ceph.com/issues/23140

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3d447dbf2befc1fab7b8888eb670f9f2dfe5555a)

7 years agotests: cleanup: drop upgrade/jewel-x/point-to-point-x 20641/head
Nathan Cutler [Thu, 1 Feb 2018 22:53:29 +0000 (23:53 +0100)]
tests: cleanup: drop upgrade/jewel-x/point-to-point-x

This subsuite is only for testing upgrades from one jewel
point release to another. In luminous and master it serves no
useful purpose.

Fixes: http://tracker.ceph.com/issues/22888
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit bc9b2e6c898d495b4eb3304b6128dd1292a212cc)

Conflicts:
    qa/suites/upgrade/jewel-x/point-to-point-x/point-to-point-upgrade.yaml
       This file has different content than in master, due to the following
       three luminous-only fixes:
           61e4de133d9fa1eddcd858152b7828515e0b280f
           e50af8bb04bd37bd0e28ddc7935e29bcdc106b05
           ee02b5c037c67c708335745898328e990ecacc90

7 years agoMerge pull request #20584 from pdvian/wip-23113-luminous
Nathan Cutler [Tue, 27 Feb 2018 16:08:00 +0000 (17:08 +0100)]
Merge pull request #20584 from pdvian/wip-23113-luminous

luminous: doc: Add missing pg states from doc

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years ago12.2.4 v12.2.4
Jenkins Build Slave User [Mon, 26 Feb 2018 21:47:18 +0000 (21:47 +0000)]
12.2.4

7 years agoMerge pull request #20596 from ceph/backport-20592
Alfredo Deza [Mon, 26 Feb 2018 21:45:32 +0000 (16:45 -0500)]
Merge pull request #20596 from ceph/backport-20592

luminous: ceph-volume: add ANSIBLE_SSH_RETRIES=5 to functional tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20597 from ceph/backport-20594
Alfredo Deza [Mon, 26 Feb 2018 21:41:27 +0000 (16:41 -0500)]
Merge pull request #20597 from ceph/backport-20594

luminous: ceph-volume: log the current running command for easier debugging

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20283 from dzafman/wip-22911
David Zafman [Mon, 26 Feb 2018 19:56:00 +0000 (11:56 -0800)]
Merge pull request #20283 from dzafman/wip-22911

luminous: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoceph-volume: log the current running command for easier debugging 20597/head
Andrew Schoen [Mon, 26 Feb 2018 16:04:07 +0000 (10:04 -0600)]
ceph-volume: log the current running command for easier debugging

Fixes: http://tracker.ceph.com/issues/23004
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 911ca889cde3983b138317314d79c89cffdc649f)

7 years agoceph-volume: add ANSIBLE_SSH_RETRIES=5 to functional tests 20596/head
Andrew Schoen [Mon, 26 Feb 2018 13:54:56 +0000 (07:54 -0600)]
ceph-volume: add ANSIBLE_SSH_RETRIES=5 to functional tests

Occasionally we get tests that fail because a test node becomes
'unreachable'. This should avoid those ssh connection issues we see sometimes
by increasing the amount of times ansible will try to reconnect to a node
after an ssh failure.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 44a5b2eb3adb18f9534ede267c181b7e5f053a2c)

7 years agoMerge pull request #20566 from theanalyst/wip-luminous-23108
Abhishek L [Mon, 26 Feb 2018 07:03:51 +0000 (08:03 +0100)]
Merge pull request #20566 from theanalyst/wip-luminous-23108

luminous: rgw: parse old rgw_obj with namespace correctly

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #20565 from theanalyst/wip-luminous-23106
Abhishek L [Mon, 26 Feb 2018 06:59:49 +0000 (07:59 +0100)]
Merge pull request #20565 from theanalyst/wip-luminous-23106

luminous: rgw: use explicit index pool placement

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20564 from theanalyst/wip-luminous-frontend-fix
Abhishek L [Mon, 26 Feb 2018 06:58:19 +0000 (07:58 +0100)]
Merge pull request #20564 from theanalyst/wip-luminous-frontend-fix

luminous: rgw:  make init env methods return an error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20563 from smithfarm/wip-23103-luminous
Abhishek L [Mon, 26 Feb 2018 06:57:20 +0000 (07:57 +0100)]
Merge pull request #20563 from smithfarm/wip-23103-luminous

luminous: tools: ceph-disk: v12.2.2 unable to create bluestore osd using ceph-disk

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #20515 from cbodley/wip-23072
Abhishek L [Mon, 26 Feb 2018 06:56:27 +0000 (07:56 +0100)]
Merge pull request #20515 from cbodley/wip-23072

luminous: cmake: check bootstrap.sh instead before downloading boost

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agorgw: parse old rgw_obj with namespace correctly 20586/head
Yehuda Sadeh [Tue, 13 Feb 2018 18:18:08 +0000 (10:18 -0800)]
rgw: parse old rgw_obj with namespace correctly

Fixes: http://tracker.ceph.com/issues/22982
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit c598e53c95ec09772353a12e9162464ccb76e0a8)

7 years agodoc: Add missing pg states from doc 20584/head
David Zafman [Tue, 20 Feb 2018 22:44:56 +0000 (14:44 -0800)]
doc: Add missing pg states from doc

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 058d8b007df3c13749dd4393cd4633b360677528)

7 years agoMerge pull request #20501 from dzafman/wip-23054
David Zafman [Sat, 24 Feb 2018 17:34:59 +0000 (09:34 -0800)]
Merge pull request #20501 from dzafman/wip-23054

luminous: Snapset inconsistency is detected with its own error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20548 from dzafman/wip-23093
David Zafman [Sat, 24 Feb 2018 17:34:17 +0000 (09:34 -0800)]
Merge pull request #20548 from dzafman/wip-23093

luminous: last-stat-seq returns 0 because osd stats are cleared

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #20405 from liewegas/wip-osd-new-class-luminous
Yuri Weinstein [Fri, 23 Feb 2018 22:55:01 +0000 (14:55 -0800)]
Merge pull request #20405 from liewegas/wip-osd-new-class-luminous

luminous: mon: accept crush device class to osd new

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #20353 from adamemerson/wip-22603-22604-luminous
Yuri Weinstein [Fri, 23 Feb 2018 20:51:53 +0000 (12:51 -0800)]
Merge pull request #20353 from adamemerson/wip-22603-22604-luminous

luminous: Backport of cache manipulation: issues #22603 and #20353

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #20568 from ceph/backport-20513
Alfredo Deza [Fri, 23 Feb 2018 18:37:55 +0000 (13:37 -0500)]
Merge pull request #20568 from ceph/backport-20513

luminous: ceph-volume warn on mix of filestore and bluestore flags

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20266 from linuxbox2/luminous-mploc-22655
Abhishek L [Fri, 23 Feb 2018 17:01:25 +0000 (18:01 +0100)]
Merge pull request #20266 from linuxbox2/luminous-mploc-22655

luminous: rgw: return valid Location element, CompleteMultipartUpload

Reviewed-By: Radoslaw Zarzynski <rzarzynski@redhat.com>
7 years agoMerge pull request #20236 from linuxbox2/luminous-rgw-22121
Abhishek L [Fri, 23 Feb 2018 17:00:28 +0000 (18:00 +0100)]
Merge pull request #20236 from linuxbox2/luminous-rgw-22121

luminous: rgw: URL-decode S3 and Swift object-copy URLs

Reviewed-By: Radoslaw Zarzynski <rzarzynski@redhat.com>
7 years agoMerge pull request #19104 from nickjanus/cors_not_found
Abhishek L [Fri, 23 Feb 2018 16:59:33 +0000 (17:59 +0100)]
Merge pull request #19104 from nickjanus/cors_not_found

luminous: rgw: Fix S3 compatibility bug when CORS is not found

Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>
7 years agoceph-volume tests verify exclude_group_options works with groups of flags 20568/head
Alfredo Deza [Tue, 20 Feb 2018 19:38:11 +0000 (14:38 -0500)]
ceph-volume tests verify exclude_group_options works with groups of flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 11ba052010fe9a5c78c359bd05aa3042a69a575c)

7 years agoceph-volume tests add exclusion checks in prepare
Alfredo Deza [Mon, 19 Feb 2018 21:31:29 +0000 (16:31 -0500)]
ceph-volume tests add exclusion checks in prepare

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 43be2aafc187892a1ff61039dabea9efe0c9b1b3)

7 years agoceph-volume tests add exclusion checks in create
Alfredo Deza [Mon, 19 Feb 2018 21:31:04 +0000 (16:31 -0500)]
ceph-volume tests add exclusion checks in create

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2f6248c94be9b57766b820b23826714897153a51)

7 years agoceph-volume util.arg_validators create an arg group exclusion check
Alfredo Deza [Mon, 19 Feb 2018 21:28:52 +0000 (16:28 -0500)]
ceph-volume util.arg_validators create an arg group exclusion check

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dec6e086cf221e9b90f1b227aa63717d387f9f8e)

7 years agoceph-volume lvm.create use the new group option exclution
Alfredo Deza [Mon, 19 Feb 2018 21:26:36 +0000 (16:26 -0500)]
ceph-volume lvm.create use the new group option exclution

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f6ac244bb51742429c01f5d8726cbe86210f4266)

7 years agoceph-volume lvm.prepare use the new group option exclution
Alfredo Deza [Mon, 19 Feb 2018 21:26:21 +0000 (16:26 -0500)]
ceph-volume lvm.prepare use the new group option exclution

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e6b3c46368ea3e12fcebfef8fa60dd0824f3e1b4)

7 years agoceph-volume lvm.common remove --journal-size as it is not used/supported
Alfredo Deza [Mon, 19 Feb 2018 20:13:04 +0000 (15:13 -0500)]
ceph-volume lvm.common remove --journal-size as it is not used/supported

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3400f31787c197c52037f26a69fa1e2c1a6c294f)

7 years agoceph-volume lvm.common define arg groups for objectstores
Alfredo Deza [Mon, 19 Feb 2018 20:10:47 +0000 (15:10 -0500)]
ceph-volume lvm.common define arg groups for objectstores

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3e7c995ebe252cb785da12548113ac855aa5219b)

7 years agorgw: parse old rgw_obj with namespace correctly 20566/head
Yehuda Sadeh [Tue, 13 Feb 2018 18:18:08 +0000 (10:18 -0800)]
rgw: parse old rgw_obj with namespace correctly

Fixes: http://tracker.ceph.com/issues/22982
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit c598e53c95ec09772353a12e9162464ccb76e0a8)

7 years agorgw: use explicit index pool placement 20565/head
Yehuda Sadeh [Wed, 7 Feb 2018 21:23:36 +0000 (13:23 -0800)]
rgw: use explicit index pool placement

Fixes: http://tracker.ceph.com/issues/22928
If explicit placement is set, it overrides placement_target.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5f32134f6d6bafa2ee6c948804a66cbe310b7876)

7 years agoMerge pull request #20559 from ceph/backport-wip-rm23080
Andrew Schoen [Fri, 23 Feb 2018 14:34:10 +0000 (15:34 +0100)]
Merge pull request #20559 from ceph/backport-wip-rm23080

luminous ceph-volume correctly normalize mount flags

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-disk: erase 110MB for nuking existing bluestore 20563/head
Kefu Chai [Mon, 12 Feb 2018 10:02:30 +0000 (18:02 +0800)]
ceph-disk: erase 110MB for nuking existing bluestore

10M is not enough for erasing the data partition *and* db partition,
let's make it 110M.

Fixes: http://tracker.ceph.com/issues/22354
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 627475d679bd588d29323ff81fdf90499ae55d0e)

7 years agoceph-volume tests.util update tests for mount flags 20559/head
Alfredo Deza [Thu, 22 Feb 2018 18:47:53 +0000 (13:47 -0500)]
ceph-volume tests.util update tests for mount flags

Include parametrized flags for ensuring a combination of values will
still be normalized regardless on how they are on ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6671cb4fd1636505b1a10d81d3bd0db23bb2fbd0)

7 years agoceph-volume util.prepare normalize mount flags for CLI
Alfredo Deza [Thu, 22 Feb 2018 18:47:11 +0000 (13:47 -0500)]
ceph-volume util.prepare normalize mount flags for CLI

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6bd226a3b4d37627bf9f039133de7792af3e9710)

7 years agoMerge pull request #20545 from ceph/backport-20537
Alfredo Deza [Thu, 22 Feb 2018 23:44:56 +0000 (18:44 -0500)]
Merge pull request #20545 from ceph/backport-20537

luminous: ceph-volume: adds support to zap encrypted devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agotest: Fail upon flush_pg_stats timeout 20548/head
David Zafman [Mon, 29 Jan 2018 19:06:52 +0000 (11:06 -0800)]
test: Fail upon flush_pg_stats timeout

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agomgr/ClusterState: record osd_stat for out osds too
Sage Weil [Thu, 10 Aug 2017 19:00:46 +0000 (15:00 -0400)]
mgr/ClusterState: record osd_stat for out osds too

I'm not quite sure why we were doing this. :/

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a7612d33d1e4184af7faa6712fc50d6487a09c40)

7 years agorgw: Add cache introspection and manipulation 20353/head
Adam C. Emerson [Sat, 27 Jan 2018 21:18:53 +0000 (16:18 -0500)]
rgw: Add cache introspection and manipulation

Provide the ability to examine and delete elements from the cache.

Fixes: http://tracker.ceph.com/issues/22603
Fixes: http://tracker.ceph.com/issues/22604
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 69fafcaaed6c5ee32f12ac857dbf4af7e13483f8)
Conflicts:
Resolved by removing a few C++17isms.
Make list helper public to deal with CentOS 7 GCC Bug

7 years agoceph-volume: use lv.encrypted property with lvm zap 20545/head
Andrew Schoen [Thu, 22 Feb 2018 17:01:10 +0000 (11:01 -0600)]
ceph-volume: use lv.encrypted property with lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 64659882d78ad4376573fc21ef5107dfb324bba2)

7 years agoceph-volume: adds an encrypted property to api.lvm.Volume
Andrew Schoen [Thu, 22 Feb 2018 17:00:15 +0000 (11:00 -0600)]
ceph-volume: adds an encrypted property to api.lvm.Volume

This property will return a boolean indicating if the lv
is encrypted or not.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5f1c0219d048ca507329e8eb66f47cd1c8c31d4e)

7 years agoceph-volume: add zapping to lvm dmcrypt tests
Andrew Schoen [Wed, 21 Feb 2018 22:03:03 +0000 (16:03 -0600)]
ceph-volume: add zapping to lvm dmcrypt tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1d5606885a8b1e42a4530b1a525f0273c09e5761)