rgw: read bucket owner when following olh if pending entries

We need to read the bucket owner, so that we'd be able to apply any
changes to the object. Bucket owner is needed for quota related
accounting.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove warning

This cleanup is not essential at this point

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: clean up compilation warnings

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: implement dump() and generate test instances

Add missing methods for some cls_rgw op types.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: guard certain operations using olh tag

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: improve olh atomicity

Add olh_tag attr to olh object, and use it instead of obj_tag as object
data may be colocated with the olh. Use the olh_tag to guard various
operations.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: encode timestamp in pending olh info

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: remove instance entry when removing delete marker

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: send x-amz-version-id and x-amz-delete_marker header fields

send these fields on obj delete as defined by the api. This required
adding an index read so that we can identify whether an instance is
actually a delete marker.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add dump_string_header()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: add missing cls_cxx_create()

This was found during code read, not necessarily a real issue, however,
we counted on the object being created implicitly by setting xattrs on
it. Instead make sure the object is created.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: fix update of olh to reflect non existing object

need to update olh appropriately when removing the last reference of an
object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: inc olh epoch when updating log

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix access to object through the null instance

Beforehand we were following olh if the explicit 'null' instance was
specified.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: adjust versioning enable/suspend api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: propagate object owner and mtime for deletion marker

deletion marker needs to have object owner and mtime properties.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: multiple changes related to obj removal

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: send "null" version id if needed

on bucket list versions

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formatter: no need for dynamic allocation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

osd: fix filter_prefix scoping in omap_get_vals

filter_prefix should only affect the case where it's greater than
start_after, otherwise we'll end up with a key equals to start_after.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: add bi put

An admin operation to set bucket index entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw, rgw-admin: create bi list operation

Can now list bucket index entries that correspond to a single key. This
is useful when debugging versioned objects issues.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw, rgw-admin: move bi_get() entry encoding to cls

Create a new structure to hold entry info, so that we can later creata a
bi_list() operation.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: add missing flags encoding to rgw_bucket_dir_entry::dump()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

common, rgw: json escaping gets input buf size

We now deal with strings that embed a null character.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw-admin, cls_rgw: add bi_get objclass operation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: keep null-versioned object as versioned object

when olh is created. Mark the object as such, so that we'll know to
handle it correctly. This is needed so that we could handle the 'null'
version as required.
Still doesn't work correctly.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: follow olh where needed

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: unlink obj instance

use new objclass method

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: unlink object instance

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: update olh log when unlinking entry

Need to update log so that later client will be able to appropriately
remove the object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: more cleanup

move versioned data and olh handling into class methods.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: more cleanup

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: reorganize rgw_bucket_link_olh()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: convert RGWRados::get_attr() to new interface

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: switch get_obj_iterate() to new interface

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: purge intent log

We don't use the intent log anymore, just remove it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: adjust to new interfaces

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: s/RGWRados::ObjectCtx/RGWObjectCtx

Can't forward declare nested class.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: clean up system obj interfaces

Create a new structure to use, RGWRados::SystemObject, similar to the
previous changes made for the data objects.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: change RGWRados::get_attr()

This is an interim fix, will need to make it work similarly to the other
calls.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rework prepare_get_obj(), get_obj()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove more unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix missing state initalization

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: switch RGWRados::delete_obj() to new interface

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove old index update calls

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: get rid of put_obj_meta(), replace with put_system_obj()

Last use of put_obj_meta() was for system object, simplify it for that
specific use case and rename.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't use put_system_obj() for data objects

Use the new interface instead.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: pass around object context refrences, remove unused code

We want to make it clear that the object context is required for
non-system objects, pass it by reference (rather than by pointer). Make
the put processors get it as a constructor param and keep it on the
object.
Still haven't updated everything, as some stuff is going to change
anyway.
Remove the plain put processor, as it is not used anymore.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: remove plain object processor

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: start reorganizing RGWRados

Need to reduce the various states of the different operations. The aim
is to separate the system objects operations from the data objects
operations, and compartmentalize the various systems.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: s/RGWRadosCtx/ObjectCtx

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move RGWRadosCtx into RGWRados

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: set olh if object has been versioned

Bucket could have versioning suspended, but object has been versioned,
so once object has olh pointer in it, we need to continue maintainig it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: allow olh linking to null instance objects

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: don't remove olh objects

A single head object can contain both olh data, and object data. Don't
remove object if it has olh information on it, just reset the other
info.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: initial versioned object removal implementation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: bucket versioning status is tri-state

disabled, enabled, suspended.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add support for version-id-marker

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: cls_bucket_list returns raw key in map

This is needed so that we dump the objects in the required order (from
newest to oldest versions for each object). While we're at it, revert to
the original encoding of rgw_bucket_dir, which will be better for
backward compatibility.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: update the appropriate prev key entry

We need to update the previous key entry that it's no longer the latest,
need to use the list entry, not the data entry (which only serves as a
pointer to the list entry).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix rgw_obj initialization

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: request state and various op functionality use rgw_obj_key

instead of plain string. This is needed so that we use the object
instance in the processing.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cleanup, get rid of req_state::object

We don't want this extra memory allocation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: restful api now dumps versions

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: propagate dirent flags to rgw (from cls), other fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: deletion marker needs to keep instance entry

Otherwise we wouldn't be able to find it when we remove it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: list object versions is optional

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: bucket listing gets extra param for versioning

Can specify whether we want to see versions or not.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: revise the data model

We don't need to keep a listing entry for the olh.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add get_type() to rgw ops

This is needed so that we can identify ops correctly.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add 'versioning', and 'versions' to handled subresources

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: object manifest should reflect instance

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update json encoding for rgw_obj

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: follow olh if needed

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: olh trim op is read/write

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: tie set_olh() to object completion

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: more atomicity fixes, set_olh()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: guard against racing writes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: olh atomicity groundwork

still more to do

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw, cls_rgw: trim olh log functionality

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: apply olh log functionality

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: olh init op

Need an op to initialize olh when a new olh is created. It will set its
initial version and tag.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw-admin: add olh readlog command

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: implement rgw_bucket_olh_log_entry::dump()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: new api to retrieve olh log

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: bucket index link olh interface

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: object instance olh linking

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: bucket index link olh

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: encode / decode obj and list index keys

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rgw: prepare groundwork for olh

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: obj_stat() follows on olh

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: some code cleanup

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: init olh tag

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: gen rand lowercase string (stl string version)

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: adjust return code when generating random strings

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: gen rand lower alphanumberic string

add a function to generate a random lowercase string, don't want other
characters.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>