Adam Kupczyk [Fri, 5 Aug 2016 12:21:51 +0000 (14:21 +0200)]
Got rid of most bare pointers => now mostly unique_ptr.
Fixed bugs that appeared on unit tests. Added unit tests.
These are created to improve code coverage achieved using s3-tests.
Added comments and improved formatting.
Fixed improper processing when last 0-15 bytes are decoded alone.
Fixed problem with PUT with x-amz-copy-source when source object is compressed.
Fixed problem with not decrypting data on S3 copy operation
Fixed problem with syncing when object is compressed and encrypted at the same time.
Fixed retrieving original length when compression is on.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
Adam Kupczyk [Fri, 29 Jul 2016 14:09:02 +0000 (16:09 +0200)]
Introduced plugins for crypto and isal implementation.
Added submodule for isa-l_crypto
Fixed problem with uninitialised var.
Enable to use crypto plugin if available.
Now isal crypto plugin compiles only on intel.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
Adam Kupczyk [Fri, 10 Jun 2016 14:05:00 +0000 (16:05 +0200)]
Added aws:kms mode to get/put/post operations.
Refactored to export selection of encryption to rgw/rgw_crypt.h/cc
Added quick exit to from_base64 when string_ref is empty.
Fixed POST logic, so aws encryption can also be applied to objects created from web forms
Extended KeystoneService with barbican user configuration.
Keystone provides token, Barbican provides secret.
Added option rgw_barbican_url.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com> Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Introduced AES_256_CTR block encryptor.
Added filters for encryptors to RGWPostObj and RGWPutObj.
Exposed cur_stripe from RGWObjManifest::obj_iterator
Moved RGWPutObjProcessor_Multipart from rgw_op.cc to rgw_rados.h.
Implemented actual AES256_CTR and AES256_ECB modes.
Refactored RGWGetObj_BlockDecrypt to process multipart manifest differently.
Initial work of server-side-encryption customer key (Amazon SSE-C).
Initial work of "auto encryption".
Add prepare_encryption method to RGWInitMultipart operation.
Changed naming of client encryption mode to SSE-C-AES256.
Eliminated AWS encryption related headers from being stored with object's metadata.
Added set_attr utility functions.
split RGW_ATTR_CRYPT_KEY => RGW_ATTR_CRYPT_KEYSEL + RGW_ATTR_CRYPT_KEYID
Added interfaces create_random_key_selector and get_actual_key_from_kms
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com> Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Sage Weil [Tue, 4 Apr 2017 17:33:04 +0000 (13:33 -0400)]
os/bluestore: fsck: verify blob.unused field
Two checks:
- verify that no logical extents reference the portion of the
blob marked unused, and
- verify that the csum (if present) is zero for any unused
region.
rgw: switch to std::array in RGWBulkUploadOp due to C++11 and FreeBSD.
Before this patch RGWBulkUploadOp::terminal_errors was declared as
std::initializer_list<int>. Unfortunately, a constexpr constructor
for it is available since C++14, not C++11. This was causing build
failures on FreeBSD.
Sage Weil [Tue, 4 Apr 2017 15:52:24 +0000 (11:52 -0400)]
os/bluestore: do not extend blobs with unused bitmap
If we resize the blob we need to adjust the resolution of the
unused bitmap, and that is only possible for some bit patterns.
For now just ignore blobs with unused blocks.
Add an assert in add_tail() so that we don't forget that
add_tail is (probably) where we'd would do that adjustment.
Amir Vadai [Wed, 22 Mar 2017 10:41:12 +0000 (12:41 +0200)]
msg/async/rdma: Make port number an attribute of the Connection not of the Device
Since multiple connections on different ports could exist, shouldn't use
device->active_port, instead use conn->ibport.
Or in other words, now Device object doesn't have an active_port,
instead every port specific action (create_qp, get_lid, get_gid etc.)
need to specify the port number. The information about the port number
is known to the connection (RDMAConnectedSocket*) who is the caller of
those actions.
Issue: 995322
Change-Id: I482cb87c04ba99845dc44f6dd0547835fe814ebf Signed-off-by: Amir Vadai <amir@vadai.me>
Erwan Velu [Fri, 31 Mar 2017 12:54:33 +0000 (14:54 +0200)]
ceph-disk: Adding retry loop in get_partition_dev()
There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/<device>.
It appear that waiting a very short is usually enough to get the partition beein populated.
Analysis:
update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and
the actual creation on the /sys/block/<device>/* entrypoint.
On our CI that race occurs pretty often but trying to reproduce it locally never been possible.
This patch is almost a workaround rather than a fix to the real problem.
It offer retrying after a very short to be make a chance the device to appear.
This approach have been succesful on the CI.
Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs.
A typical output from the build running on a CI with that code.
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda
get_partition_dev: Found partition 2 for /dev/sda after 1 tries
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid
common/simple_spin: use __ppc_yield() on all powerpc archs
__ppc_yield() is declared in sys/platform/ppc.h by glibc, for better
portability we just use the inlined assembly here. the shared resource
hints are supported by PowerPC ISA 2.06 but on older PowerPC cores, they
are no-ops. so it's fine to do this way.
Sage Weil [Mon, 27 Mar 2017 15:32:03 +0000 (11:32 -0400)]
os/bluestore: flush osr on collection split
We need to ensure that any preceding txcs have finished their deferred
writes before splitting, or else later txcs on the child sequencer will
not order/wait for deferred writes correctly before deallocated extents.
Fixes: http://tracker.ceph.com/issues/19379 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log