rgw: handle errors via exceptions in RGWStreamIOEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: cleanup in the LoadGen frontend.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port the LoadGen frontend to the new RGWStreamIOEngine infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: ONLY improve formatting in the FCGI frontend.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port the FCGI frontend to the new RGWStreamIOEngine infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port the ASIO frontend to the new RGWStreamIOEngine infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: cleaning in the glue code for CivetWeb.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: switch out RGWCivetWeb to the new frontend infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOChunkingEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOReorderingEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOConLenControllingEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: ONLY rename RGWMongooseFrontend to RGWCivetWebFrontend.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: ONLY rename RGWMongoose to RGWCivetWeb and marks overrides.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: eradicate dynamic memory allocations from RGWEnv.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOBufferingEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWClientIO::get_env() is pure virtual now.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOAccountingEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWMongoose uses RGWStreamIOEngine instead of RGWStreamIO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement RGWStreamIOLegacyWrapper for using new front-ends with RGWStreamIO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: dissect non-core ops from RGWStreamIOEngine to RGWStreamIOFacade.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: dissect accounting interface from RGWClientIO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add basic infrastructure for decorators over RGWStreamIOBase.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rename RGWStreamIOBase to RGWStreamIOEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: further cleaning in RGWStreamIOBase.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: strip out AWSv4 signature calculation from RGWStreamIOBase.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve const-correctness in the front-end infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: move transfer accounting management from RGWClientIO to RGWStreamIO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce RGWStreamIOBase to separate all non-IO related things.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: mark asio frontend as experimental

Users will need to set "rgw-asio-frontend" as an experimental
feature.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: s3 PutObj sends status first

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: s3 GetObj sends status first

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: asio frontend uses async_read

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: implement ClientIO interface for asio + beast

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: asio frontend calls process_request

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWEnv::set takes boost::string_ref

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: implement Frontend interface for asio

Signed-off-by: Casey Bodley <cbodley@redhat.com>

vstart: add --rgw_frontend option

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add skeleton for boost::asio frontend

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add submodule for Beast header-only library

Beast is a c++ http protocol library build on top of boost::asio

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #11534 from ceph/wip-17607-ceph-disk

tests: ceph-disk: force debug monc = 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11512 from wjwithagen/wip-wjw-freebsd-depends

FreeBSD: update require packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11204 from liewegas/wip-rados-ack

librados: do not request osd ack if no completed completion is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #11213 from liewegas/wip-osd-fastinfo

osd: add a pg _fastinfo attribute to reduce per-io metadata updates

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #11354 from liewegas/wip-rocksdb

rocksdb: update to latest, and make it the default for the mons

Merge pull request #11500 from liewegas/wip-lttng

CMakeLists.txt: enable LTTNG by default

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11540 from liewegas/wip-bluestore-cow

os/bluestore: default bluestore_clone_cow=true

Merge pull request #11539 from liewegas/wip-bluestore-gc

os/bluestore: no garbage collection for uncompressed blobs

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: default bluestore_clone_cow=true

This has been stable for a long time.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "ceph_test_objecstore: use 1TB block device"

This reverts commit 5389133ff9a7783d77796ed73893e26f34c6e5d2.

This is too slow with the bitmap allocator.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: No garbage collection for uncompressed blobs

Signed-off-by: Roushan Ali<roushan.ali@sandisk.com>

Merge pull request #11536 from liewegas/wip-fullbit-dump

mds: fix EMetaBlob::fullbit xattr dump

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11533 from ifed01/wip-kstore-rmcoll-fix

os/kstore: rmcoll fix to satisfy store_test

Reviewed-by: Sage Weil <sage@redhat.com>

mds: fix EMetaBlob::fullbit xattr dump

- show xattr names!
- bound the values

Signed-off-by: Sage Weil <sage@redhat.com>

tests: ceph-disk: force debug monc = 0

The sh function will collect both stderr and stdout and debug
will mess the json parsing.

Fixes: http://tracker.ceph.com/issues/17607
Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11525 from wjwithagen/wip-wjw-freebsd-getopt

tools/ceph-monstore-update-crush.sh: FreeBSD getopt is not compatible…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11524 from wjin/filestore_perfcounter

os/filestore: clean filestore perfcounters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11518 from umullan/uday

doc: fix broken links

Merge pull request #11522 from wjwithagen/wip-wjw-freebsd-readable.sh

FreeBSD:test/encoding/readable.sh": fix nproc and ls -v calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11528 from jcsp/wip-vstart-1osd

vstart: decrease pool size if <3 OSDs

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PG: add perfcounters for pg info updates

Note info updates (of any kind), fastinfo updates,
and biginfo updates.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: proper set resulting pnext to avoid returning local var

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/kstore: fix KStore rm_coll handler to behave similar to bluestore one

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

ceph-objectstore-tool: relax cur_struct_v checks

All compat pg formats can handle past interval removal
and mark-complete.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: make fastinfo apply conditionally

This way we don't have to bother deleting the fastinfo
attr, which will reduce the amount of work the
underlying store has to do.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add _fastinfo PG attr for common pg_info_t updates

For most IO operations we only update a handful of fields
in the pg_info_t structure.  However, the full struct,
when encoded, is on the order of 800 bytes.

This adds a new attribute, _fastinfo, which contains only
the most commonly updated fields.  When present, the
fastinfo fields should be overlayed on top of the full
info struct contained in the existing info attr.  If
a field outside of the "fast" set is updated, we clear
the fastinfo attribute and update the full info attr.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: pass last_writen_info to _prepare_write_info

We'll use this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: operator== for pg hit sets, history, info

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop unused args to _prepare_write_info

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: pg_stat_t ondisk_log_size == log_size

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10325 from snakeAngel2015/rgw-support-s3subuser

rgw: add suport for creating S3 type subuser of admin rest api
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

os/kstore: uniform Collection implementation with BlueStore one

Signed-off-by:Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11481 from ddiss/cmake_rgw_deps

cmake: check WITH_RADOSGW for fcgi and expat dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: decrease pool size if <3 OSDs

Makes it easy to do OSD=1 for e.g. doing cephfs
stuff where we don't care much about the OSDs
behaviour.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10239 from weiqiaomiao/wqm-wip-setolh

rgw: should assign 'olh_bl" to state.attrset[RGW_ATTR_OLH_ID_TAG] instead of 'bl'
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11324 from yehudasa/wip-14589

rgw: dump objects in RGWBucket::check_object_index()
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

tools/ceph-monstore-update-crush.sh: FreeBSD getopt is not compatible, use the one from packages

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11432 from hansbogert/patch-3

doc: Update install-ceph-gateway.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

os/filestore: clean filestore perfcounters

Move filestore perfcounters from objectstore to filestore itself
and change internal index name.

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

Merge pull request #11027 from liewegas/wip-denc

fast denc encoding

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>

Merge remote-tracking branch 'gh/kraken'

cmake: find GIT_VER variables if there is no .git dir

find CEPH_GIT_VER & CEPH_GIT_NICE_VER if there is no
ceph/.git dir by parsing ceph/src/.git_version.

scenario is for building from source tarballs generated
by make-dist, since make-dist generates .git_version
by running make_version, and then packs .git_version
into the tarball.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit c5c6b59f296f0ace3a6cfd939027118e82ab50a4)

Merge pull request #11502 from liewegas/wip-bluestore-keybug

os/bluestore: fix escaping of chars > 0x80

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/kstore: fix escaping of chars > 0x80 in keys

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix escaping of odd chars >0x80 in keys

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: test unprintable chars

Signed-off-by: Sage Weil <sage@redhat.com>

FreeBSD:test/encoding/readable.sh": fix nproc and ls -v calls

 Neither of these is available in FreeBSD.
 But sort -n is Posix, and does the same ATM.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11521 from markhpc/wip-send_inline

msg/async: set ms_async_send_inline to false to improve small randread iops

Set ms_async_send_inline to false to improve small randread iops.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #11262 from jcsp/wip-17270-master

Revert "osdc: After write try merge bh."

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

11.0.2

Merge pull request #11211 from ceph/wip-no-recordlock-test

test: temporarily remove fork()ing flock tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

Merge pull request #11440 from ceph/wip-jlayton-linkref

client: don't take extra target inode reference in ll_link

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11421 from ceph/wip-jlayton-cephfs

Small pile of random cephfs fixes and cleanup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11515 from badone/wip-find_best_info-typos

osd: Fix typos in PG::find_best_info

Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>

osd: Fix typos in PG::find_best_info

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #11513 from badone/clear_primary_state-typo

osd: fix typo in PG::clear_primary_state

Merge pull request #11514 from xiexingguo/xxg-wip-bluestore-1015

os/bluestore: formatting nits

Reviewed-by: Sage Weil <sage@redhat.com>