Merge pull request #9143 from tchaikov/wip-osdmaptool-man

man: osdmaptool: document test-map-pgs* options

Reviewed-by: Vikhyat Umrao <vumrao@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9153 from ceph/openstack_wusui

OpenStack setup scripts.

Merge pull request #8669 from dreamhost/wip-15532

rgw/s3website: ErrorDoc & Double-fault support

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9150 from stiopaa1/compatset_removeIncludeVector_mc

common: CompatSet.h: remove unneeded #include<vector>

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9158 from cxwshawn/wip-tc-fix

osd: erase duplicated header include

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9169 from stiopaa1/osd_hitSetRemoveFormatterInclude

osd/HitSet: remove unneeded #include - Formatter

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9186 from stiopaa1/osd_fixTypoInComment

osd/OSD.h: fix typo in comment

osd/OSD.h: fix typo in comment

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

xio: rename enqueue_for_send() to enqueue()Reviewed-by: Sage Weil <sage@redhat.com>

xio: rename enqueue_for_send() to enqueue()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9181 from dillaman/wip-rbd-mirror-test

qa/workunits/rbd: fixed rbd_mirror teuthology runtime errors

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: fixed rbd_mirror teuthology runtime errors

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Add date to README

Merge pull request #8307 from songbaisen/song16

mon: paxos small optimization on store_state

Reviewed-by: Kefu Chai <kchai@redhat.com>

Pipe: take a ref to existing while we are waiting

Pipe: take a ref to existing while we are waiting

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9159 from tchaikov/wip-kill-warnings

osd,auth: silence "-Wmisleading-indentation" warnings

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

journal: live replay might skip entries from previous object setReviewed-by: Mykola Golub <mgolub@mirantis.com>

journal: live replay might skip entries from previous object set

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9092 from trociny/wip-15746

doc: update mirroring guide to include pool/image status commands

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: rbd-mirror daemon stress test

This test repeatedly runs rbd bench-write, kills the process
randomly to create an unclean journal shutdown, and verifies
that the image content replicates correctly.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: skip partially complete tag entries during playback

If a journal client does not fully write out its buffered entries
before quiting, replay should skip over all remaining out-of-
sequence entries for the tag.

Fixes: http://tracker.ceph.com/issues/15864
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: helper method to detect newer tags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: close, advance, and open object set ordering

Flush in-flight appends to open objects before advancing the
active object set.  Additionally, don't start recording to the
new objects until after advancing the active set.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: new ObjectRecorder closed callback

The callback will be invoked if there were in-flight appends
when the close was requested.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: do not flag append as full if already known to be full

Once an object has overflowed or a close is in-progress, re-attempting
to close the object is not needed since the async process is already
underway.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: delay object overflow event until in-flight appends settled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: ignore flush on closed/overflowed object

The journal would be in-progress on transitioning to a new
object recorder in a newer object set.  Once the records
re-attach to the new object player they will automatically
flush.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: implicitly detach future's flush handler on append

If the future is already in-flight, there is no purpose served
by requesting the future be flushed to disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: async callback for advancing the active object set

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: re-fetch active object before advancing set during replay

During a live replay, it's possible that an append and and overflow
into the next object could race with the live playback of the same
object.  Re-fetch an "empty" object at least once before advancing
to next set to ensure all records have been read.

Fixes: http://tracker.ceph.com/issues/15665
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8747 from jcsp/wip-15599

client: report root's quota in statfs (#15599)

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #8582 from ukernel/wip-trim-purging

MDS: don't pin purging denties

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9166 from chhabaramesh/master

os/bluestore/BitmapFreelistManager: fix length calculation

Reviewed-by: Sage Weil <sage@redhat.com>

fixing wrong length cal in BitmapFreelist

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

man: osdmaptool: document test-map-pgs* options

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8350 from songbaisen/song17

mon: do not alloc monmap on heap in Elector::dispatch()

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw/s3website: whitespace style fixes

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

rgw/s3website: Fix ErrocDoc memory leak.

Yehuda pointed [1] out a memory leak with the RGWGetObj not being handed back, and
suggested that shared_ptr was used to handle it, rather than the prior approach
of explicitly calling handler->put_op.

This commit implements just that change, in isolation from other changes, for
ease of comparision testing.

X-Reference-1: https://github.com/ceph/ceph/pull/8669#discussion_r63445516
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #9026 from badone/wip-log-syslog-info

log: Log.cc: Assign LOG_INFO priority to syslog calls

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8727 from liewegas/wip-collection-attr

osd: remove collection attr methods from ObjectStore interface

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7638 from asheplyakov/filejournal-open-ret

FileJournal::_open: return the correct error code

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #9098 from theanalyst/rgw/bucket-unused-ioctx

rgw_rados: remove unused ioctx for domain root in create_bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7306 from branch-predictor/bp-fix-osdmap-loaderr

OSD/OSDService: make osdmap load error more human-understandable

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7118 from efirs/ef_rwlock_for_index_manager

os/fileStore: Replace Mutex with RWLock in IndexManager

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7163 from tchaikov/wip-pr-6085

mon: use CephOsdName for osd args instead of CephInt

Reviewed-by: Sage Weil <sage@redhat.com>

utime.h: fix timezone issue in round_to_* funcs.Reviewed-by: Samuel Just <sjust@redhat.com>

utime.h: fix timezone issue in round_to_* funcs.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8259 from theanalyst/rgw/fix-15234

rgw: return -EACCESS for system requests also

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8892 from flyd1005/wip-boost-starts_with

rgw: cleanup: use boost::algorithm::starts_with to replace our own implementations

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd/HitSet: remove unneeded #include - Formatter

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9109 from cbodley/wip-rgw-dup-zone

rgw: radosgw-admin rejects duplicate zone names in a zonegroup
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

client: report root's quota in statfs

When user is mounted a quota-restricted inode
as the root, report that inode's quota status
as the filesystem statistics in statfs.

This allows us to have a fairly convincing illusion
that someone has a filesystem to themselves, when
they're really mounting a restricted part of
the larger global filesystem.

Fixes: http://tracker.ceph.com/issues/15599
Signed-off-by: John Spray <john.spray@redhat.com>

osd,auth: silence "-Wmisleading-indentation" warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

OSD: erase duplicated header include
especially osd_types.h, maybe affect compile speed.
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

test/objectstore/CMakeLists.txt: fix unittes_bit_alloc typoReviewed-by: Josh Durgin <jdurgin@redhat.com>

test/objectstore/CMakeLists.txt: fix unittes_bit_alloc typo

Merge pull request #9118 from oritwas/wip-rgw-apache-realm-pull

rgw: fix realm pull and period pull for apache frontend

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

test/objectstore/CMakeLists.txt: fix unittes_bit_alloc typo

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #9152 from liewegas/wip-bitmap-allocator

os/bluestore: bitmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>

OpenStack setup scripts.

A set of scripts used to bring up an OpenStack instance on one site,
connect a ceph cluster, add pools to store OpenStack glance images,
cinder volumes, nova vms, and cinder backups on rbd, and bring up a nova
compute node.

os/bluestore: bitmap allocator

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: pass size to allocator

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: add config for allocator

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com

Merge pull request #7787 from wjwithagen/patch-1

clang: common/config.cc basename() requires libgen.h includes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8689 from runsisi/wip-fix-ev-sk

msg/async/EventSocket: fix missing break statement

Reviewed-by: Haomai Wang <yuyuyu101@163.com>
Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>

Merge pull request #8717 from ZhenyuLeng/noneed_op_is_discardable

osd: make op_is_discardable check optional

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8961 from xiexingguo/xxg-wip-fixepochtrunc

osd: don't get inc osdmap name truncated

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8985 from badone/wip-iops-typo

common: obj_bencher.cc add missing ":"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9011 from stiopaa1/osdmap_removeUnneededIncludeFiles

osd/OSDMap.h: remove unneeded include files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9075 from runsisi/wip-compute-cost

osd/ReplicatedBackend: reuse the already computed cost

Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9077 from liewegas/wip-osd-map-cache

osd: instrument map cache hits and misses

Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>

Merge pull request #9078 from liewegas/wip-clean-temps

mon, osdmap: optimize check and cleanup of pg_temp for new osdmaps

Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8298 from wjwithagen/wip-wjw-freebsd-autobuild-2

build: many fixes for freebsd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8480 from songbaisen/song19

crush: When consider rule size use the correct method.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8974 from liewegas/wip-addr-cleanup

msg/msg_types: clean up entity_addr_t

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9027 from roidayan/make-debs_avoid_creating_recursive_conf_dir

make-debs.sh: avoid creating recursive conf dir

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9144 from vumrao/wip-vumrao-15899

doc : adding explanation of --no-increasing for

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8452 from jcsp/wip-15266

pybind: fix unicode handling in CephFSVolumeClient::purge

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc : adding explanation of --no-increasing for
reweight-by-utilization and test-reweight-by-utilization
commands

Fixes: http://tracker.ceph.com/issues/15899
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

doc/changelog: missed v10.2.1 changelog

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9141 from liewegas/wip-10.2.1

doc/release-notes: v10.2.1 release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/release-notes: v10.2.1 release notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9135 from flaf/master

doc: improvement of the "tunables" part

Reviewed-by: Sage Weil <sage@redhat.com>

doc: improvement of the "tunables" part

* Some CRUSH_TUNABLES$N references were false.
* The explanations concerning the warning when tunables are
  non-optimal were false or confusing.
* Make the definition of the "default" profile clearer.

Signed-off-by: François Lafont <francois.lafont@ac-versailles.fr>

The function op_is_discardable introduces lock overhead which can
The function op_is_discardable introduces lock overhead which can
not be ignored.It would be better to judge if to discard op when
the connection disconnected as a parameter whether the network is
believable or extremely unlikelihood.

add g_conf osd_discard_disconnected_ops to control whether to
discard op when the connection disconnected
Waiting for test

Signed-off-by: ZhenyuLeng <lengzhenyu@ict.ac.cn>

Merge pull request #9124 from stiopaa1/osd_someMorePrivate

osd/OSD.h: move some data members under private

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9128 from stiopaa1/osd_constOperatorPGQueueable

osd/OSD: pass op() params by const in PGQueueable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9131 from stiopaa1/compatset_removeConstructor

common: CompatSet.h: Remove unneeded constuctor

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9132 from shinobu-x/wip-shinobu

cls_rbd: fix comment typo

Reviewed-by: Sage Weil <sage@redhat.com>

src/cls/rbd/cls_rbd.cc: fix super trivial typo

CompatSet.h: remove unneeded #include<vector>

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

CompatSet.h: Remove unneeded constuctor

We have a constuctor taking a std::string so there is no need for
the constuctor taking char*.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

doc: update mirroring guide to include pool/image status commands

Fixes: http://tracker.ceph.com/issues/15746
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd/OSD: pass op() params by const in PGQueueable

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #8767 from jjhuo/org_jianjian

.mailmap and .orgmap: organization affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd/OSD.h: move some data members under private

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9123 from idryomov/wip-qa-rbd-dont-chown

qa: rbd: don't modprobe, chown sysfs files or udevadm settle

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa: rbd/simple_big.sh: drop extraneous sudo

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa: rbd/concurrent.sh: suppress rbd map output

rbd_map_image() echoes the id of the image and is used in command
substitution:

    id=$(rbd_map_image "${image}")

Output from rbd map isn't consumed and clobbers the return.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa: rbd: don't modprobe, chown sysfs files or udevadm settle

This is a followup to 38a572011dbb ("qa: rbd/map-snapshot-io.sh: don't
chown sysfs files") for the rest of the workunits:

- /sys/bus/rbd/add_single_major is used if present, so chown'ing
  /sys/bus/rbd/add doesn't help.  Use sudo on rbd map instead.

- Don't modprobe or udevadm settle - rbd CLI tool takes care of that.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

os: remove collection attr methods

These have been deprecated since hammer, and were only kept so that you
could do an offline upgrade from firefly.  Bye bye!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop pre-jewel pg attr method

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9070 from liewegas/doc-ct

doc: words of caution about cache tiering

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Nick Fisk <nick@fisk.me.uk>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rados: adjust example cache tiering settings

These are saner values.

Signed-off-by: Sage Weil <sage@redhat.com>