mgr/dashboard: taskmanager: support exception handler

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot rollback implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: major refactoring for using the Task decorator

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: restcontroller: support for detail methods

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Task decorator for controller endpoints

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot edit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot edit implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: use dashboard pool API for creating pools

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: image edit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image edit implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: snapshot create/delete tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: snapshot create/delete implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: image deletion test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image delete implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: restcontroller: allow POST with empty body

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: restcontroller: allow DELETE to return JSON body

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image disk usage implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: asynchronous image create test

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: asynchronous image create

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: rbd: new image list format

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: rbd: image list implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: added script to execute backend API requests

Example of usage:

```
$ ./run-backend-api-request.sh POST /api/rbd '{"name": "myimage2", "pool_name": "rbd", "size": 53687091200, "obj_size": 4194304, "features": ["exclusive-lock", "journaling", "layering"], "stripe_unit": null, "stripe_count": null, "data_pool": null}'

$ ./run-backend-api-request.sh GET /api/rbd
```

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: viewcache: remove VALUE_EXCEPTION

Instead of returning the exception object as a result of a function
decorated with `@ViewCache()`, raise the exception in the function
caller thread.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21416 from tchaikov/wip-tox-pwd

test/dashboard: specify workdir using tox.ini

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #18932 from yehudasa/wip-rgw-cloud-sync

rgw cloud sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21356 from smithfarm/wip-23657

rpm,debian: Ensure all ceph-disk runtime dependencies are declared for ceph-base

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

test/dashboard: specify workdir using tox.ini

--workdir was introduced in tox v2.4.0, but the tox shipped by
ubuntu/xenial is 2.3.1. also because i removed the step to prepare the
tox using "pip install -r requirements.txt", what we have is the tox
v2.3.1 . so, instead of passing workdir in the command line, we specify
this setting in tox.ini.

also drop the tox in requirements.txt, it's just misleading. and we are
not using the tox installed in the venv.

Fixes: http://tracker.ceph.com/issues/23709
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21342 from SUSE/wip-argparse-fix-validate_command-error

pybind/ceph_argparse: Fix UnboundLocalError if command doesn't validate

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21331 from liewegas/wip-slow-ops

qa/suites/rados/monthrash: whitelist SLOW_OPS

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21317 from liewegas/wip-23622

common/Formatter: fix string_view usage for {json,xml}_stream_escaper

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21312 from wjwithagen/wjw-PG-public

osd: make PG::*Force* event structs public

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21293 from tchaikov/wip-no-venv-for-dashboard

test: only test enabled python bindings

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21287 from yanghonggang/master

os/kstore: fix statfs problem and add vstart.sh support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21174 from guzhongyan/crush-log-cleanup

crush: cleanup update_device_class() log messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21110 from majianpeng/fix-objecter

osdc: dec num_in_flight for pool_dne case.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #21029 from rzarzynski/wip-osd-close_op_ctx-nohobjcpy

osd: don't memcpy hobject_t in PrimaryLogPG::close_op_ctx().

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21018 from ashitakasam/wip-bug-23440

osd: remove duplicated "commit_queued_for_journal_write" in OpTracker

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20670 from chardan/jfw-wip-rand_migration-core

Extends random.h: numeric types relaxed to compatible types (with

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21350 from tspmelo/notifications-improvement

mgr/dashboard: Notification improvements

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com

Merge pull request #20996 from rzarzynski/wip-bs-bug23246

os/bluestore: fix exceeding the max IO queue depth in KernelDevice.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #21345 from ceph/fix_e2fsprogs

specs: require of e2fsprogs

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #20658 from majianpeng/cmake-link-error

os/CMakeLists: fix link errro when enable WITH_PMEM=ON.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21332 from cbodley/wip-cmake-lz4

cmake: enable LZ4 by default

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw/tests_http_manager: fix initialization

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

json_formattable: fix out of bounds array entity removal

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #21181 into master

* refs/pull/21181/head:
fuse: wire up fuse_ll_access

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21276 into master

* refs/pull/21276/head:
MDSMonitor: fix mds health printed in bad format

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #21323 into master

* refs/pull/21323/head:
mds: properly dirty sessions opened by journal replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: force last writer wins on marker writes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

doc/radosgw: cloud sync docs

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #21362 from dzafman/wip-hex-digest

osd: Change shard digests to hex like object info digests

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: rename aws tier type to 'cloud'

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: etag fixes

Use string instead of bufferlist to avoid potential issues.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fixes following code review

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw/test_multi: fix a few tests to only iterate over rw zones

Some of the tests require at least two read-write (regular rgw) zones

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw: zone_cloud: deal with key representation and other fixes

Needed to present a key to the tests that reflected its original name
and version_id (and etag), so that the callers don't need to be modified.
However, this can only be achieved if we get the key, which doesn't work
if the caller was just listing the bucket objects. Created a new CloudKey
class to deal with the different issues there.
Also, other test related fixes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: streaming put also stores content_type and other fields

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix parse_tier_config_param function

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

rgw: don't call http data callbacks under lock

There is no need to hold req_data->lock when calling into client
callbacks. This removes an unneeded lock dependency (that is a
problem when cancelliing coroutines stack).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: coroutines: cancel stacks on teardown

If we don't cancel stacks, ops might not be destructed, so ops callbacks
could still be active.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't store etag with extra null character at the end

head objects etag attr doesn't need to store an extra null char.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync: store source object info in destination object

store extra meta params on target object (original name, version_id, etag,
etc.)

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync: store versioned epoch in target object

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw: initial work on cloud sync test

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: api adjustment following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rework cloud sync configuration

Change the configuration structure to separate between
connections and acl mappings, and other changes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fixes following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use different io id when waking up interval wait

Can't use a zero channel id as a wildcard.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix compilation warning

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

vstart: fixes for mstart

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: apply acl mappings

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: sync acls

still not transforming acls according to configuration

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: acl_mappings config handling

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: remove bucket suffix configurable

No need to have a special suffix, target path can be used instead.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: use configurable target path

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: dump config

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formatter: add two utility classes

For automatic section scoping

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: configure different targets

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: new config structure, support multiple connections

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: more fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sync module tier config fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync, sync object meta

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: adjust array access interface

appending new entry by providing empty index, setting existing entries
by providing negative index.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: formattable, set, erase, unitest

Extend the formattable api to provide a mechanism to set and erase
entities by a string key that references them. E.g., "foo.bar[123].xyz"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: handle the situation that get a 409 response from S3 correctly
pass the http body and use it when creating a bucket

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: avoid use Chunked transfer encoding in RGW's request

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: add a bucket-suffix tier-config

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: virtual hosted-style support

add host-style field in tier-config to specify the related
zone's hosted-style used in request from RGW, if this config
is not specified, path hosted-style will be used as default.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: RGWRESTStreamRWRequest::send_prepare(), url encode dest

encode the resource in send_prepare(), as a result, all the callers
to do_send_prepare() have do url_encode before calling it.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: add encode_slash param in url_encode

when sent request to S3, we should not encode the forward
slash character('/') in the object key name, so we need add
a encode_slash param in url_encode to decide whether to encode
the slash or not.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: use GMT time format defined by RFC1123 in http request header

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: sync modules: adapt to formattable based config

still need to fix rgw-admin

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/ceph_json_formattable: add unitest

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: formattable fix, api changes

add implicit casts

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: refine Formattable api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sync modules, use JSONFormattable

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rgw_str_to_bool() should also accept "true"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

common/formattable: initial work

initial work on a dict-like api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #21026 from ivancich/wip-rgw-unordered-listing

rgw: ability to list bucket contents in unsorted order for efficiency

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>