Merge pull request #11175 from asheplyakov/wip-rbd-default-features

librbd: parse rbd_default_features config option as a string

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11567 from ceph/wip-master-rgw-0lenfix2

Don't loop forever when reading data from 0 sized segment.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Don't loop forever when reading data from 0 sized segment.

The 0 sized segment can arise depending on how
the client uploads the object in the first place.
The cpu loop then happens during a swift `GET'.

Signed-off-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #11541 from stiopaa1/mds_sessionmap_addConst

mds/SessionMap: add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

unittest_bluestore_types: add random put_ref test case

This appeared to fail under load, but works fine here.
:/

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_bluestore_types: fix gc tests

In ef8989ce7417034aa715de4fe83d1f0363c044f4 we disabled
gc for non-compressed blobs.  So mark these test ones as
compressed.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11438 from trociny/wip-17488

rbd-nbd: support disabling auto-exclusive lock transition logic

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11546 from umullan/uday

doc: Fix broken link for caps

Merge pull request #11558 from ifed01/wip-rmcoll-more-fixes

os/memstore, os/filestore: fix store_test's to satisfy rm_coll behavi…

Reviewed-by: Sage Weil <sage@redhat.com>

os/memstore, os/filestore: fix store_test's to satisfy rm_coll behavior expected in store_test

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11207 from yehudasa/wip-17372

rgw: get_zonegroup() uses "default" zonegroup if empty
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #10919 from pritha-srivastava/wip-17076

rgw: Replacing '+' with "%20" in canonical uri for s3 v4 auth.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11316 from stiopaa1/mds_sessionMapUnnamedNamespace

mds/SessionMap.cc:put classes in unnamed namespace

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11450 from stiopaa1/mds_damagetable_moveClassesToCCfile

mds/DamageTable: move classes to .cc file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11511 from stiopaa1/mds_mdsmap_addConstToMemFunctions

mds/MDSMap.h: add const to member functions

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11550 from renhwztetecs/renhw-wip-mgr-doc

doc/mgr: fix mgr how long to wait to failover

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11304 from ajarr/wip-volume-client-recovery-fixes

ceph_volume_client: fix recovery from partial auth update

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11386 from xiexingguo/xxg-wip-mgr-1008

mgr: misc minor changes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11507 from ceph/wip-17579

build/ops: deb: add python dependencies where needed

Reviewed-by: Alfredo Deza <adeza@redhat.com>

build/ops: deb: add python dependencies where needed

this reverts 48f4576 and f140a8c

Fixes: http://tracker.ceph.com/issues/17579
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

build/ops: deb: put depends on separate lines

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #11557 from xiexingguo/xxg-wip-bluestore-1019

os/bluestore: kill overlay related options

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11516 from david-z/wip-add-force-ro-ceph-string

common/ceph_string: add ceph string constants for CEPH_SESSION_FORCE_RO

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11552 from tchaikov/wip-fix-enc

include/object: pass "snapid_t&" to bound_encode()

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11434 from batrick/mds-cmd-deprecated

Flag deprecated mds commands and omit deprecated mon commands in help output

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11486 from batrick/client-gid

client: get gid from MonClient

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11373 from ukernel/wip-mds-recall-warning

mds: fix false "failing to respond to cache pressure" warning

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11551 from wjwithagen/wip-wjw-freebsd-lltng

do_freebsd.sh: Do not use LTTNG on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

include/object: pass "snapid_t&" to bound_encode()

otherwise the STL container's encoders will segfault when trying to
figure out the element size: they use "denc(*(A*)nullptr, elem_size)"
to do the calculation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/mgr: fix mgr how long to wait to failover

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

do_freebsd.sh: Do not use LLTNG on FreeBSD

  FreeBSD has Dtrace

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11534 from ceph/wip-17607-ceph-disk

tests: ceph-disk: force debug monc = 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: kill overlay related options

These two options are not used anymore.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: Fix broken link for caps
http://tracker.ceph.com/issues/17587
Signed-off-by: Uday udaymjl@gmail.com

Merge pull request #11512 from wjwithagen/wip-wjw-freebsd-depends

FreeBSD: update require packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11204 from liewegas/wip-rados-ack

librados: do not request osd ack if no completed completion is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #11213 from liewegas/wip-osd-fastinfo

osd: add a pg _fastinfo attribute to reduce per-io metadata updates

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #11354 from liewegas/wip-rocksdb

rocksdb: update to latest, and make it the default for the mons

Merge pull request #11500 from liewegas/wip-lttng

CMakeLists.txt: enable LTTNG by default

Reviewed-by: Kefu Chai <kchai@redhat.com>

mds/SessionMap: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #11540 from liewegas/wip-bluestore-cow

os/bluestore: default bluestore_clone_cow=true

Merge pull request #11539 from liewegas/wip-bluestore-gc

os/bluestore: no garbage collection for uncompressed blobs

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: default bluestore_clone_cow=true

This has been stable for a long time.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "ceph_test_objecstore: use 1TB block device"

This reverts commit 5389133ff9a7783d77796ed73893e26f34c6e5d2.

This is too slow with the bitmap allocator.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: No garbage collection for uncompressed blobs

Signed-off-by: Roushan Ali<roushan.ali@sandisk.com>

Merge pull request #11536 from liewegas/wip-fullbit-dump

mds: fix EMetaBlob::fullbit xattr dump

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11533 from ifed01/wip-kstore-rmcoll-fix

os/kstore: rmcoll fix to satisfy store_test

Reviewed-by: Sage Weil <sage@redhat.com>

mds: fix EMetaBlob::fullbit xattr dump

- show xattr names!
- bound the values

Signed-off-by: Sage Weil <sage@redhat.com>

tests: ceph-disk: force debug monc = 0

The sh function will collect both stderr and stdout and debug
will mess the json parsing.

Fixes: http://tracker.ceph.com/issues/17607
Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #11525 from wjwithagen/wip-wjw-freebsd-getopt

tools/ceph-monstore-update-crush.sh: FreeBSD getopt is not compatible…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11524 from wjin/filestore_perfcounter

os/filestore: clean filestore perfcounters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11518 from umullan/uday

doc: fix broken links

Merge pull request #11522 from wjwithagen/wip-wjw-freebsd-readable.sh

FreeBSD:test/encoding/readable.sh": fix nproc and ls -v calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11528 from jcsp/wip-vstart-1osd

vstart: decrease pool size if <3 OSDs

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PG: add perfcounters for pg info updates

Note info updates (of any kind), fastinfo updates,
and biginfo updates.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: proper set resulting pnext to avoid returning local var

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/kstore: fix KStore rm_coll handler to behave similar to bluestore one

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

ceph-objectstore-tool: relax cur_struct_v checks

All compat pg formats can handle past interval removal
and mark-complete.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: make fastinfo apply conditionally

This way we don't have to bother deleting the fastinfo
attr, which will reduce the amount of work the
underlying store has to do.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add _fastinfo PG attr for common pg_info_t updates

For most IO operations we only update a handful of fields
in the pg_info_t structure.  However, the full struct,
when encoded, is on the order of 800 bytes.

This adds a new attribute, _fastinfo, which contains only
the most commonly updated fields.  When present, the
fastinfo fields should be overlayed on top of the full
info struct contained in the existing info attr.  If
a field outside of the "fast" set is updated, we clear
the fastinfo attribute and update the full info attr.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: pass last_writen_info to _prepare_write_info

We'll use this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: operator== for pg hit sets, history, info

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop unused args to _prepare_write_info

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: pg_stat_t ondisk_log_size == log_size

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10325 from snakeAngel2015/rgw-support-s3subuser

rgw: add suport for creating S3 type subuser of admin rest api
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

os/kstore: uniform Collection implementation with BlueStore one

Signed-off-by:Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11481 from ddiss/cmake_rgw_deps

cmake: check WITH_RADOSGW for fcgi and expat dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: decrease pool size if <3 OSDs

Makes it easy to do OSD=1 for e.g. doing cephfs
stuff where we don't care much about the OSDs
behaviour.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10239 from weiqiaomiao/wqm-wip-setolh

rgw: should assign 'olh_bl" to state.attrset[RGW_ATTR_OLH_ID_TAG] instead of 'bl'
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11324 from yehudasa/wip-14589

rgw: dump objects in RGWBucket::check_object_index()
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

tools/ceph-monstore-update-crush.sh: FreeBSD getopt is not compatible, use the one from packages

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11432 from hansbogert/patch-3

doc: Update install-ceph-gateway.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

os/filestore: clean filestore perfcounters

Move filestore perfcounters from objectstore to filestore itself
and change internal index name.

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

Merge pull request #11027 from liewegas/wip-denc

fast denc encoding

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>

Merge remote-tracking branch 'gh/kraken'

cmake: find GIT_VER variables if there is no .git dir

find CEPH_GIT_VER & CEPH_GIT_NICE_VER if there is no
ceph/.git dir by parsing ceph/src/.git_version.

scenario is for building from source tarballs generated
by make-dist, since make-dist generates .git_version
by running make_version, and then packs .git_version
into the tarball.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit c5c6b59f296f0ace3a6cfd939027118e82ab50a4)

Merge pull request #11502 from liewegas/wip-bluestore-keybug

os/bluestore: fix escaping of chars > 0x80

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/kstore: fix escaping of chars > 0x80 in keys

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix escaping of odd chars >0x80 in keys

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: test unprintable chars

Signed-off-by: Sage Weil <sage@redhat.com>

FreeBSD:test/encoding/readable.sh": fix nproc and ls -v calls

 Neither of these is available in FreeBSD.
 But sort -n is Posix, and does the same ATM.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11521 from markhpc/wip-send_inline

msg/async: set ms_async_send_inline to false to improve small randread iops

Set ms_async_send_inline to false to improve small randread iops.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

mgr: shutdown finisher in a graceful way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/MgrMonitor: make period of sending digests configurable

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/MgrMonitor: break loop on catching specified standby

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/MgrMonitor: unify updating map code path

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/MgrStandby: do not ignore result of init() of monc

Which can fail due to missing keyring.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/MgrStandby: use mgr_beacon_period to send beacon

Fixes: http://tracker.ceph.com/issues/17455
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11262 from jcsp/wip-17270-master

Revert "osdc: After write try merge bh."

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

11.0.2

Merge pull request #11211 from ceph/wip-no-recordlock-test

test: temporarily remove fork()ing flock tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

Fixes: #17587
doc: Fixing the broken link which was refactored for authentication as part of other PR
Signed-off-by: uday udaymjl@gmail.com

Merge pull request #11440 from ceph/wip-jlayton-linkref

client: don't take extra target inode reference in ll_link

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11421 from ceph/wip-jlayton-cephfs

Small pile of random cephfs fixes and cleanup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

common/ceph_string: add ceph string constants for CEPH_SESSION_FORCE_RO

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

rbd: add possibility to set rbd_default_features config option via string

Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>

config: thread safe options

For now there are two ways options: 1) direct reference (cct->_conf->foo),
2) registering the observer. The former way is obviously not thread safe,
and the latter is a bit cumbersome. Hence the new get_val() method which
takes a lock and returns a *copy* of the value. Thus it's safe to modify
the option in question at the run time (when threads are already running).
Such options should be declared with SAFE_OPTION macro and can be accessed
only via get_val()/set_val() methods and observers, attempt to reference
them directly (cct->_conf->safe_opt) won't compile.

Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>