cls/refcount: add chunk_refcount

refcount() for chunked object is added (based on source id,
pool id and offset)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #21416 from tchaikov/wip-tox-pwd

test/dashboard: specify workdir using tox.ini

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #18932 from yehudasa/wip-rgw-cloud-sync

rgw cloud sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21356 from smithfarm/wip-23657

rpm,debian: Ensure all ceph-disk runtime dependencies are declared for ceph-base

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

test/dashboard: specify workdir using tox.ini

--workdir was introduced in tox v2.4.0, but the tox shipped by
ubuntu/xenial is 2.3.1. also because i removed the step to prepare the
tox using "pip install -r requirements.txt", what we have is the tox
v2.3.1 . so, instead of passing workdir in the command line, we specify
this setting in tox.ini.

also drop the tox in requirements.txt, it's just misleading. and we are
not using the tox installed in the venv.

Fixes: http://tracker.ceph.com/issues/23709
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21342 from SUSE/wip-argparse-fix-validate_command-error

pybind/ceph_argparse: Fix UnboundLocalError if command doesn't validate

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21331 from liewegas/wip-slow-ops

qa/suites/rados/monthrash: whitelist SLOW_OPS

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21317 from liewegas/wip-23622

common/Formatter: fix string_view usage for {json,xml}_stream_escaper

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21312 from wjwithagen/wjw-PG-public

osd: make PG::*Force* event structs public

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21293 from tchaikov/wip-no-venv-for-dashboard

test: only test enabled python bindings

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21287 from yanghonggang/master

os/kstore: fix statfs problem and add vstart.sh support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21174 from guzhongyan/crush-log-cleanup

crush: cleanup update_device_class() log messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21110 from majianpeng/fix-objecter

osdc: dec num_in_flight for pool_dne case.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #21029 from rzarzynski/wip-osd-close_op_ctx-nohobjcpy

osd: don't memcpy hobject_t in PrimaryLogPG::close_op_ctx().

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21018 from ashitakasam/wip-bug-23440

osd: remove duplicated "commit_queued_for_journal_write" in OpTracker

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20670 from chardan/jfw-wip-rand_migration-core

Extends random.h: numeric types relaxed to compatible types (with

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #21350 from tspmelo/notifications-improvement

mgr/dashboard: Notification improvements

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com

Merge pull request #20996 from rzarzynski/wip-bs-bug23246

os/bluestore: fix exceeding the max IO queue depth in KernelDevice.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #21345 from ceph/fix_e2fsprogs

specs: require of e2fsprogs

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #20658 from majianpeng/cmake-link-error

os/CMakeLists: fix link errro when enable WITH_PMEM=ON.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21332 from cbodley/wip-cmake-lz4

cmake: enable LZ4 by default

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw/tests_http_manager: fix initialization

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

json_formattable: fix out of bounds array entity removal

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge PR #21181 into master

* refs/pull/21181/head:
fuse: wire up fuse_ll_access

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21276 into master

* refs/pull/21276/head:
MDSMonitor: fix mds health printed in bad format

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #21323 into master

* refs/pull/21323/head:
mds: properly dirty sessions opened by journal replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: force last writer wins on marker writes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

doc/radosgw: cloud sync docs

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #21362 from dzafman/wip-hex-digest

osd: Change shard digests to hex like object info digests

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: rename aws tier type to 'cloud'

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: etag fixes

Use string instead of bufferlist to avoid potential issues.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fixes following code review

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw/test_multi: fix a few tests to only iterate over rw zones

Some of the tests require at least two read-write (regular rgw) zones

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw: zone_cloud: deal with key representation and other fixes

Needed to present a key to the tests that reflected its original name
and version_id (and etag), so that the callers don't need to be modified.
However, this can only be achieved if we get the key, which doesn't work
if the caller was just listing the bucket objects. Created a new CloudKey
class to deal with the different issues there.
Also, other test related fixes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: streaming put also stores content_type and other fields

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix parse_tier_config_param function

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

rgw: don't call http data callbacks under lock

There is no need to hold req_data->lock when calling into client
callbacks. This removes an unneeded lock dependency (that is a
problem when cancelliing coroutines stack).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: coroutines: cancel stacks on teardown

If we don't cancel stacks, ops might not be destructed, so ops callbacks
could still be active.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't store etag with extra null character at the end

head objects etag attr doesn't need to store an extra null char.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync: store source object info in destination object

store extra meta params on target object (original name, version_id, etag,
etc.)

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync: store versioned epoch in target object

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/rgw: initial work on cloud sync test

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: api adjustment following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rework cloud sync configuration

Change the configuration structure to separate between
connections and acl mappings, and other changes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fixes following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use different io id when waking up interval wait

Can't use a zero channel id as a wildcard.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix compilation warning

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

vstart: fixes for mstart

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: apply acl mappings

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: sync acls

still not transforming acls according to configuration

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: acl_mappings config handling

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: remove bucket suffix configurable

No need to have a special suffix, target path can be used instead.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: use configurable target path

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: dump config

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formatter: add two utility classes

For automatic section scoping

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: configure different targets

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: aws sync: new config structure, support multiple connections

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: more fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sync module tier config fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cloud sync, sync object meta

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

formattable: adjust array access interface

appending new entry by providing empty index, setting existing entries
by providing negative index.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: formattable, set, erase, unitest

Extend the formattable api to provide a mechanism to set and erase
entities by a string key that references them. E.g., "foo.bar[123].xyz"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: handle the situation that get a 409 response from S3 correctly
pass the http body and use it when creating a bucket

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: avoid use Chunked transfer encoding in RGW's request

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: add a bucket-suffix tier-config

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: virtual hosted-style support

add host-style field in tier-config to specify the related
zone's hosted-style used in request from RGW, if this config
is not specified, path hosted-style will be used as default.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: RGWRESTStreamRWRequest::send_prepare(), url encode dest

encode the resource in send_prepare(), as a result, all the callers
to do_send_prepare() have do url_encode before calling it.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: add encode_slash param in url_encode

when sent request to S3, we should not encode the forward
slash character('/') in the object key name, so we need add
a encode_slash param in url_encode to decide whether to encode
the slash or not.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: use GMT time format defined by RFC1123 in http request header

Signed-off-by: lvshanchun <lvshanchun@gmail.com>

rgw: sync modules: adapt to formattable based config

still need to fix rgw-admin

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

test/ceph_json_formattable: add unitest

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: formattable fix, api changes

add implicit casts

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: refine Formattable api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: sync modules, use JSONFormattable

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rgw_str_to_bool() should also accept "true"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

common/formattable: initial work

initial work on a dict-like api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #21026 from ivancich/wip-rgw-unordered-listing

rgw: ability to list bucket contents in unsorted order for efficiency

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #21291 from smithfarm/wip-23596

RGWCivetWeb::read_data: fix arguments to mg_read() call

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21285 from joke-lee/wip-max_keys_convert

rgw: return EINVAL if max_keys can not convert correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21129 from oritwas/wip-rgw-admin-no-cache

rgw: radosgw-admin should not use metadata cache for readonly commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20834 from linuxbox2/wip-linked

rgw_file: conditionally unlink handles when direct deleted

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20926 from yehudasa/wip-22963-truncation

rgw: fix for usage truncated flag

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #20814 from niupengju/master

rgw: Fix multisite Synchronization failed when read and write delete …

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: Change shard digests to hex like object info digests

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #21140 from ceph/cv

ceph-volume: remove extra space

install-deps: add liblz4 to freebsd build dependencies

Signed-off-by: Casey Bodley <cbodley@redhat.com>

debian: add build dependency for lz4

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rpm: add build dependency for lz4

Signed-off-by: Casey Bodley <cbodley@redhat.com>

lz4: adjusted release notes

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: Enable LZ4 if required library is available.

Signed-off-by: Grant Slater <grant.slater@wearefriday.com>

Merge pull request #20962 from ricardoasmarques/wip-task-manager-frontend

mgr/dashboard: Asynchronous tasks (frontend)

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Notification improvements

Notification list is now using the new CdDate pipe for formating the date and
they are now sorted desc by date.

Fixed the class name of the info icon and changed the max-height so it can work
in smaller screens.

Decreased the size of the icons.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21099 from ricardoasmarques/wip-table-selection-refresh

mgr/dashboard: Update selected items on table refresh

Reviewed-by: Volker Theile vtheile@suse.com
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21268 from smithfarm/wip-doc-config-ceph

doc: some improvements to ceph-conf.rst

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: Asynchronous tasks front-end

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21320 from tchaikov/wip-rocksdb

rocksdb: sync with upstream

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21329 from liewegas/wip-23622-b

mon/ConfigKeyService: dump: print placeholder value for binary blobs

Reviewed-by: John Spray <jspray@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #21357 from gregsfortytwo/wip-doc-remove-naked-repo

doc: remove references to unversioned repository addresses

Reviewed-by: Sage Weil <sage@redhat.com>

build/ops: move parted runtime dependency to ceph-base

Since ceph-disk has been moved from "ceph-osd" to "ceph-base", all of
ceph-disk's dependencies should also be moved, provided they are not
needed by anything else.

Fixes: http://tracker.ceph.com/issues/23657
Signed-off-by: Nathan Cutler <ncutler@suse.com>