rgw: test cors option request with Access-Control-Request-Headers
test for https://github.com/ceph/ceph/pull/18556

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge pull request #190 from cbodley/wip-pr-17882

expect 400 for missing sse headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

expect 400 for missing sse headers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #171 from cbodley/wip-amz-date-precedence

check precedence of Date and X-Amz-Date in signatures

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #186 from tipabu/check-grant-ordering

Care less about ordering when checking grants

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #185 from theanalyst/lc-time-fixes

lc: Give more flexibility for LC expiration times

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Care less about ordering when checking grants

Some tests, like test_object_header_acl_grants and
test_bucket_header_acl_grants, use the same id for all permissions.
Sort() is stable, so those tests end up testing the order of acl grants
returned by Boto. Not sure, but I think this may in turn depend on the
order of HTTP headers, where the order is not significant.

Signed-off-by: Tim Burke <tim.burke@gmail.com>

lc: Give more flexibility for LC expiration times

Earlier values expected a lc debug interval of 2s, which is a pretty
short time window and can often lead to failures if the processing
didn't complete within the next day. This commit assumes the currently
configured LC debug interval of 10s, and gives time intervals using the
following logic:

Worst case:
LC start-time : 00:00
obj upload    : 00:01
LC run1       : 00:10 -> object not expired as it is only 9s old
LC run2       : 00:20 -> object will expire in this run, however we
can't exactly guess when this run will complete, for a moderate amount
of objects this can take anywhere between 1 to a max of 10s, let us give
it a wiggle room to complete around 8s, given the amount of objects in a
teuthology run, it should be mostly probable that the object is already
deleted within this time, so at 28s, we should have seen day1 objects being
expired.

Best case:
LC start-time: 00:00
obj upload : 00:09
LC run1 : 00:10
LC run2 : 00:20 -> obj expires, so elapsed time is around 11->19s (of
course it would almost close to 10s too), We should probably configure
the LC lock time to 10s as well just so as to ensure that the lock isn't
held for the default 60s in which case it is possible that the object
might expire in a time greater than the lock interval.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #173 from chuang-he/fix_decrypt

Data encryption is not follow the AWS agreement

Reviewed-by: Casey Bodley <cbodley@redhat.com>

modify test_sse_kms_method_head() for aws standard

Signed-off-by: hechuang <hechuang@xsky.com>

add test_sse_kms_read_declare() for aws standard

Signed-off-by: hechuang <hechuang@xsky.com>

rgw: Data encryption is not follow the AWS agreement

Encryption request headers should not be sent for GET requests and HEAD
requests if your object uses SSE-KMS/SSE-S3 or you’ll get an HTTP 400
BadRequest error.

Signed-off-by: hechuang <hechuang@xsky.com>

Merge pull request #179 from andrewgaul/copy-part-invalid-range

Add test for invalid copy part range

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #181 from theanalyst/lc-expir-disable

lc: set attribute as lifecycle_time for lc tests using timing ops

Reviewed-by: Casey Bodley <cbodley@redhat.com>

lc: add attr lifecycle_expiration for lc tests using expiration

Allows us to additionally filter out these tests which currently fail in
the teuthology runs

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #180 from theanalyst/lc-filter

lifecycle: add test cases using filter & no id

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

lifecycle: add test cases using filter & no id

Added test cases for LC policies with no ID in xml, no prefix and an
empty and non empty Filter.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Add test for invalid copy part range

References kahing/goofys#212.

Signed-off-by: Andrew Gaul <andrew@gaul.org>

Merge pull request #159 from theanalyst/wip-tagging

add tests for tagging functionality

add tests for get/put/delete/post object tagging

- adds test for put obj with tagging query
- adds test for HEAD obj with x-amz-tagging-count verification
- adds test for put object tagging
- adds test for get object tagging
- adds test for post object tagging
- adds test for delete object tagging
- also verify& exceed the max tags
- add tagging condition to bucket acl

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #167 from joke-lee/PolicyConditionOpEndWithifExist

add test case: test set policy with condition operator end with IfExists

Reviewed-By: Adam C. Emerson <aemerson@redhat.com>

add test case: test set policy with condition operator end with IfExists

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge pull request #174 from ceph/wip-underscore-prefix

Add test for bucket list with prefix with underscore

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #170 from zhangsw/lifecycle-date

add test case for lifecycle expiration date.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #168 from zhangsw/lifecycle-expiration

modify the thread sleep time in lifecycle expiration test so that it can pass on ceph master branch.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #163 from pritha-srivastava/wip-rgw-bucket-policy-tests

rgw: Adding tests for Bucket Policy.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Add test for bucket list with prefix with underscore

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

check precedence of Date and X-Amz-Date in signatures

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #166 from cbodley/wip-get-version

clear key.version_id now that GET requests return x-amz-version-id header

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

modify the thread sleep time in lifecycle expiration test so that it can
pass on ceph master branch.

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

add test case for lifecycle expiration date.

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

clear key.version_id now that GET requests return x-amz-version-id header

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: Adding tests for Bucket Policy.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #157 from zhangsw/verionid-getobjectmeta

Add test case for getting version id when getting object and object's metadata

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #158 from zhangsw/lifecycle-noncurrent

Test case for non-current expiration，multipart upload expiration and delete marker expiration

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Test case for non-current expiration, multipart upload expiration and
delete marker expiration.

Fixes: http://tracker.ceph.com/issues/19519
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

Add test case for getting version id when getting object and object's metadata

Fixes: http://tracker.ceph.com/issues/19565
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

Merge pull request #154 from ceph/wip-pr-118

functional tests for server side encryption

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

crypto: add key headers to valid HEAD requests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Added tests for HEAD method for encrypted objects.

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Added micro instuction how to configure tests for KMS (barbican) integration

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Modified aws kms tests to use config variable kms_keyid

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

A bit more test for kms

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Additional tests for server side encryption, S3 SSE-KMS mode.
All tests belong to group 'encryption'.

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Additional tests for server side encryption, S3 SSE-C mode.
All tests belong to group 'encryption'.

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

Merge pull request #136 from jkorvin/fix-ifmodifiedsince-failed

Fix 'If-Modified-Since' format string

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #135 from jkorvin/fix

Fix If-Match test

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Fix 'If-Modified-Since' format string

According to https://tools.ietf.org/html/rfc7232
header 'If-Modified-Since' should be a correct HTTP-date.
For example: `Tue, 15 Nov 1994 12:45:26 GMT`;
not `1994-11-15T12:45:26.000Z`.

Signed-off-by: Evgenii Gorinov <korvin.lucky@gmail.com>

Fix If-Match test

According to https://tools.ietf.org/html/rfc7232, If-Match field should looks like QUOTES ETag QUOTES.

Signed-off-by: Evgenii Gorinov <korvin.lucky@gmail.com>

Merge pull request #152 from zhangsw/set-current-acl

Add a test case for setting acl on object with no version id specified.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #138 from zhangsw/lifecycle-validate

add lifecycle validate test cases.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #128 from ceph/wip-multipart-copy

Wip multipart copy

Add a test case for setting acl on object with no version id specified.

Fixes: http://tracker.ceph.com/issues/18923
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

Merge pull request #150 from ceph/wip-remove-pip

remove python2 pip

remove python2 pip

python2-pip doesn't exist in rhel and python-virtualenv ships pip in its package.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #145 from mdw-at-linuxbox/master-setupfix

Use non-broken version of setuptools, and fix bootstrap to be more portable

Use non-broken version of setuptools, and fix bootstrap to be more portable.

Most recent version of setuptools breaks when asked to load requests 0.14.0.
symptom, complains about not being able to import filterfalse thus:
        from six.moves import map, filter, filterfalse
this comes from setuptools, and older versions of setuptools don't have
this problem.

Various versions of centos7 and fedora have interesting names for packages,
centos7: python-pip is python2-pip
fedora24: python-virtualenv is python2-virtualenv
This is somewhat masked by using sudo yum: if the actual package
is installed, rpm knows that the capability is there and does nothing.
But, if the package isn't there, or you haven't chosen to set up
sudo to work that way, this does not work.

Signed-off-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #139 from linuxbox2/wip-setuptools

bootstrap:  force new setuptools

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

bootstrap:  force new setuptools

Deal setuptools upgrade problem discussed here:
 https://github.com/pypa/setuptools/issues/940

Now follows the same formula as upstream teuthology.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

add lifecycle validate test cases.

Fixes: http://tracker.ceph.com/issues/18541
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

Merge pull request #137 from ceph/wip-delete-multi-underscore

test multi-delete with object with ending underscore

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

test multi-delete with object with ending underscore

Fixes: http://tracker.ceph.com/issues/18023
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: adjust tests annotations

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #133 from ceph/wip-more-multipart-copy

more multipart copy tests

Merge pull request #132 from ceph/wip-17635

add a test for post object with empty conditions

Merge pull request #130 from ceph/wip-aws2-aws4

rgw: skip aws2 requests when testing aws4

Merge pull request #129 from ceph/wip-16930

Wip 16930

Merge pull request #117 from tianshan/fix-rgw-roundtrip

fix rgw_roundtrip test fail

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #100 from kalantal/patch-1

Update README.rst

fix broken test

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #131 from ceph/wip-put-acl-underscore

test set_acl for keys with special characters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

add a test for post object with empty conditions

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

more multipart copy tests

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test set_acl for keys with special characters

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

test concurrent bucket set acl

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #124 from tipabu/complete-multipart

Complete multi-part upload during dry-run

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Complete multi-part upload during dry-run

While it may be valuable to verify that deleting a non-existent key
is OK, that doesn't seem to be the point of this test.

Signed-off-by: Tim Burke <tim.burke@gmail.com>

rgw: adjust tests annotations

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #127 from ceph/fix_copy_obj_versioned_bucket

Fix copy obj versioned bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

more multipart copy tests

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: skip aws2 requests when testing aws4

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: skip aws2 requests when testing aws4

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #126 from ceph/rgw-lifecycle-testing

S3 lifecycle tests

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

S3 lifecycle tests

Test S3 lifecycle expiration.  This requires the test harnass code in
RGW to make sure it doesn't take days so expire anything.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

test copy object on a multipart object

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

fix test_object_copy_versioned_bucket

Do not use _create_key_with_random_content as it creates a new bucket.
Check objects content and size

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #125 from ceph/versioned_copy_obj

Add testing for copy object to/from a versioned bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Add testing for copy object to/from a versioned bucket

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #123 from ceph/get_special_key

test get object with special key name

Reviewed-by: Andrew Gaul <andrew@gaul.org>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

test get object with special key name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #119 from ceph/empty_object_range

Add range request to an empty object

Add range request to an empty object

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

fix rgw_roundtrip test fail

boto no longer rewind itself in set_contents_from_file, which will trigger the assert in
realistic/RandomContentFile::seek

Signed-off-by: Tianshan Qu <tianshan@xsky.com>

Merge pull request #116 from andrewgaul/rfc2616

Annotate tests which create non-RFC2616 headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Annotate tests which create non-RFC2616 headers

Signed-off-by: Andrew Gaul <andrew@gaul.org>

Merge pull request #115 from jmunhoz/fix-20160602

rgw: aws4: add test cases with missing signed headers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws4: fix test_object_create_missing_signed_header_aws4

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

rgw: aws4: fix test_object_create_missing_signed_custom_header_aws4

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #110 from ceph/wip-15886

rgw: add a new multipart test

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #112 from ceph/wip-add-libxslt

add missing libxml2-dev and libxslt-dev dependencies

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>