os/bluestore: batch up to bluestore_deferred_batch_ops before submitting

Allow several deferred writes to accumulate before we submit them.  In
general we have no time pressure, and on HDD (and perhaps sometimes SSD)
it is beneficial to accumulate and batch these so that they result in
fewer seeks.  On HDD, this is particularly true of seeks away from the
journal.  And on sequential workloads this can avoid seeks.  In may even
allow the block layer or SSD firmware to merge IOs and perform fewer
writes.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: only discard deallocated regions of a blob if !shared

If a blob is shared, we can't discard deallocated regions: there may
be deferred buffers in flight and we might get a read via the clone.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid waking up kv thread on deferred write completion

In a simple HDD workload with queue depth of 1, we halve our throughput
because the kv thread does a full commit twice per IO: once for the
initial commit, and then again to clean up the deferred write record. The
second wakeup is unnecessary; we can clean it up on the next commit.

We do need to do this wakeup in a few cases, though, when draining the
OpSequencers: (1) on replay during startup, and (2) on shutdown in
_osr_drain_all().

Send everything through _osr_drain_all() for simplicity.

This doubles HDD qd=1 IOPS from ~50 to ~100 on my 7200 rpm test device
(rados bench 30 write -b 4096 -t 1).

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: move many initializations into header

This is less fragile, especially with 2 constructors.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: restructure deferred write queue

First, eliminate the work queue--it's useless.  We are dispatching aio and
should not block.  And if a single thread isn't sufficient to do it, it
probably means we should be parallelizing kv_sync_thread too (which is our
only caller that matters).

Repurpose the old osr-list -> txc-list-per-osr queue structure to manage
the queuing.  For any given osr, dispatch one batch of aios at a time,
taking care to collapse any overwrites so that the latest write wins.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix OpSequencer/Sequencer lifecycle

Make osr_set refcounts so that it can tolerate a Sequencer destruction
racing with flush or a Sequencer that outlives the BlueStore instance
itself.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: move _osr_reap_done

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: reimplement/rename _sync -> _flush_all

The old implementation is racy and doesn't actually work.  Instead, rely
on a list of all OpSequencers and drain them all.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: keep all OpSequencers registered

Maintain the set of all live OpSequencers.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: keep onode refs for lifetime of obc

This ensures that we don't trim an onode from the cache while it has a
txc that is still in flight.  Which in turn ensures that if we try to read
the object, we will have any writing buffers available.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make OnodeSpace onode_map private

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make Sequencer::flush() more efficient

BlueStore collection methods only need preceding transactions to be
applied to the kv db; they do not need to be committed.

Note that this is *only* needed for collection listings; all other read
operations are immediately safe after queue_transactions().

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: add OpSequencer::drain()

Currently this is the same as flush, but more precisely it is an internal
method that means all txc's must complete.  Update _wal_apply() to use it
instead of flush(), which is part of the public Sequencer interface.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: revert throttle perfcounters

This reverts 3e40595f3cd8626cdceffa4a3a4efb088127f726

The individual throttles have their own set of perfcounters; no need to
duplicate them here.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: release deferred throttle on io finish, before cleanup

The throttle is really about limiting deferred IO; we do not need to
actually remove the deferred record from the kv db before queueing more.
(In fact, the txc that queues more will do the cleanup.)

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: separate _txc_finish_kv into _txc_{applied,committed}_kv

We can unblock flush()ing threads as soon as we have applied to the kv db,
while the callbacks must wait until we have committed.

Move methods around a bit to better match the execution order.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make flush() only wait for kv commit

The only remaining flush() users only need to see previous txc's applied
to the kv db (e.g., _omap_clear needs to see the records to delete them).

Signed-off-by: Sage Weil <sage@redhat.com>
# Conflicts:
# src/os/bluestore/BlueStore.h

os/bluestore: no need to Onode::flush() on truncate

We do not release extents until after any deferred IO, so this flush() is
unnecessary.

Signed-off-by: Sage Weil <sage@redhat.com>
# Conflicts:
# src/os/bluestore/BlueStore.cc

os/bluestore: no need to Onode::flush() in _do_read

We now ensure that deferred writes are in cache until the txc retires,
so there is no need to wait here.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: pin writing cache buffers until txc is finished

Notably, this includes WAL writes, which means an in-flight WAL write will
always be in the cache.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: write padded data into buffer cache

We rely on the buffer cache to avoid reading any deferred write data. In
order for that to work, we have to ensure the entire block whose
overwrite is deferred is in the buffer cache.  Otherwise, a write to 0~5
that results in a deferred write could break a subsequent read from 5~5
that reads the same block from disk before the deferred write lands.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: update freelist on initial commit

It does not matter if we update the freelist in the initial commit or when
cleaning up the deferred transaction; both will eventually update the
persistent kv freelist.  We maintain one case to ensure that legacy
deferred events (from a kraken upgrade) release when they are replayed.

What matters while online is the Allocator, which has an independent
in-memory copy of the freelist to make decisions.  And we can delay that
as long as we want.  To avoid any concerns about deferred writes racing
against released blocks, just defer any release until the txc is fully
completed (including any deferred writes).  This ensures that even if we
have a pattern like

 txc 1: schedule deferred write on block A
 txc 2: release block A
 txc 1+2: commit
 txc 2: done!
 txc 1: do deferred write
 txc 1: done!

then txc 2 won't do its release because it is stuck behind txc 1 in the
OpSequencer queue:

 ...
 txc 1: reaped
 txc 2: reaped (and extents released to alloc)

This builds in some delay in just-released space being usable again, but
it should be a very small amount of space relative to the size of the
store!

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: wal -> deferred

"wal" can refer to both the rocksdb wal (effectively, or journal) and the
"wal" events we include in it (mainly promises to do future IO or release
extents to the freelist).  This is super confusing!

Instead, call them 'deferred'.. deferred transactions, ops, writes, or
releases.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: larger wal device

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14030 from tchaikov/wip-denc-without-nullptr

os/bluestore: do not use nullptr to calc the size of bluestore_pextent_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12041 from yangdongsheng/rbd_mirror_clone

librbd: asynchronous clone state machine

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14058 from tchaikov/wip-doc-linkcheck

doc: add optional argument for build-doc

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14023 from dillaman/wip-rbd-coverity

librbd: fix valid coverity warnings

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14034 from liupan1111/wip-fix-comment-nbd

rbd-nbd: fix typo in comment

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-nbd: fix typo in comment.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

doc: cephfs: fix the unexpected indent warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

admin/build-doc: support optional argument for specifying sphinx builders

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13997 from tchaikov/wip-doc-fixings

doc:  fixes to silence sphinx-build

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14057 from badone/wip-RadosImport-connect

tools/rados: Check return value of connect

Reviewed-by: David Zafman <dzafman@redhat.com>

tools/rados: Check return value of connect

Fail gracefully if Rados::connect returns an error.

Fixes: http://tracker.ceph.com/issues/19319
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13971 from optimistyzy/0315_1

os/blestore/NVMEDevice: fix the I/O logic for read

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13923 from xiexingguo/wip-clean-pglog-t

OSD: drop parameter t from merge_log()

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge pull request #13938 from jimmyway/wip-chg-return-value-to-refs

osd: replace object_info_t::operator=() with decode()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13980 from majianpeng/filejournal-bufferlist-rebuild

os/filestore/FileJournal: bufferlist rebuild

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13535 from dongbula/add-rgw-finisher-to-perfcounter

rgw: add radosclient finisher to perf counter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13955 from wangzhengyong/notify_finish

rgw: handle error return value in build_linked_oids_index

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13820 from mikulely/cleanup-rgw-lc

rgw: cleanup lifecycle managament

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13481 from theanalyst/rgw/env-dout

rgw: don't log the env_map twice

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13895 from guihecheng/rgw_file-fix

rgw_file: fix reversed return value of getattr

Merge pull request #14045 from guihecheng/rgw_file-fix-retcode

rgw_file: fix non-negative return code for open operation

Merge pull request #14049 from yangdongsheng/rbd_cleanup

cleanup: rbd: fix a typo in comment

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Bluestore,NVMEDEVICE: fix the I/O logic for READ

Aio_submit will submit both aio_read/write, and also there
are synchronized read and random read, so we need to
handle the read I/O completion in a correct way.

Since random read has its own ioc, so the
num_reading for ioc will be at most 1, which will be easy
to handle in io_complete. And we need only to differentiate
whethere it is an aio_read.

Also fix the exception logic in command send, make the style
consistent.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

Merge pull request #13936 from ZVampirEM77/cleanup-rgw-doc

doc: fix typos in radosgw-admin usage

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13559 from voidbag/wip-fix-_open_super_meta

os/bluestore: fix bug in _open_super_meta()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13718 from aclamk/wip-bs-indexed-bitshift

os/bluestore: cleanup, got rid of table reference of 1<<x

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13769 from wangzhengyong/wip-noid

os/bluestore: "noid" is not always necessary in clone op

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13804 from xiaoxichen/fix_adminsocket

pybind/ceph_daemon: use small chunk for recv

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13814 from liupan1111/wip-fix-remove-when-full

rados: allow "rados purge" to delete objects when osd is full

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13822 from Liuchang0812/wip-event-center-bug

msg/async: fix crash that writing char to nonblock-fd gets EAGAIN in EventCenter::wakeup

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13853 from tchaikov/wip-19134

mon/MonClient: don't return zero global_id

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13899 from jcsp/wip-19245

mds: fix handling very fast delete ops

Merge pull request #14007 from badone/wip-api-doc-code-fixes

docs: Fix problems with example code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14028 from tchaikov/wip-kill-gcc-warning

os/bluestore: silence gcc warning

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13969 from tchaikov/wip-ceph-disk-manpage-formatting

man/8/ceph-disk: fix formatting

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw_file: fix non-negative return code for open operation

The nfs-ganesha expects a negative retcode for errors.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #13988 from guihecheng/rgw_file-fix-rename

rgw_file: fix double unref on rgw_fh for rename

Merge pull request #14033 from wjin/clean

mon/PGMonitor: rm nonused function

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

mon/PGMonitor: rm nonused function

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

os/bluestore: do not use nullptr to calc the size of bluestore_pextent_t

see also #13889, otherwise this segfaults when compiled with -O0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: silence gcc warning

src/os/bluestore/BitAllocator.h:410:8: warning: ‘virtual bool
BitMapAreaIN::child_check_n_lock(BitMapArea*, int64_t, bool)’ was
hidden    [-Woverloaded-virtual]
 bool child_check_n_lock(BitMapArea *child, int64_t required, bool
lock) {
      ^~~~~~~~~~~~~~~~~~
 /var/ceph/ceph/src/os/bluestore/BitAllocator.h:489:8: warning:   by
 ‘BitMapAreaLeaf::child_check_n_lock’ [-Woverloaded-virtual]
  bool child_check_n_lock(BitMapZone* child, int64_t required, bool
  lock);
       ^~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13917 from nikitych/add-oracle-distros

ceph-detect-init: Adds Oracle Linux Server and Oracle VM Server detect

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14006 from ShiqiCooperation/master

test/unittest_bluefs: remove unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14005 from liupan1111/wip-fix-resize-issue

rbd-nbd: update size only when NBD_SET_SIZE successful

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13922 from gregsfortytwo/wip-pg-unsigned-warning

osd: fix a signed/unsigned warning in PG

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13946 from LiumxNL/wip-170313

osd: don't share osdmap with objecter when preboot

Reviewed-by: Sage Weil <sage@redhat.com>

librbd: fix valid coverity warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13911 from liewegas/wip-bluestore-fix-flush

os/bluestore: fix bugs in bluefs and bdev flush

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12364 from dillaman/wip-rbd-mirror-notifications

rbd-mirror: replace remote pool polling with add/remove notifications

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12970 from VictorDenisov/snapshot_namespace_index

librbd: add SnapshotNamespace to ImageCtx

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14019 from singler/wip-cleanup-create_new_fs

mon/MDSMonitor: remove create_new_fs from header

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13684 from shashalu/rgw-admin-init-bucket-for-sync

rgw: fix init_bucket_for_sync retcode

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/MDSMonitor: remove create_new_fs from header

create_new_fs was refactored and moved earlier. This removes a left
over header entry.

Signed-off-by: Henrik Korkuc <henrik@kirneh.eu>

Merge pull request #14012 from optimistyzy/0317_2

bluestore,NVMEDevice: minor error for get slave core

Reviewed-by: Haomai Wang <haomai@xsky.com>

bluestore,NVMEDevice: minor error for get slave core

The second parameter should set to 1 to skip master core.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

Merge pull request #13646 from mslovy/wip-fix-test-pidfile

test: fix test_pidfile

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix typos in common/config_opts.h comments

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

rgw: kill dead lc config options

+ rgw_lifecycle_enabled is redundant with rgw_enable_lc_threads,which is
  wildly used.
+ rgw_lifecycle_thread is never used.

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

rgw: cleanup redundant file include for lc

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

docs: Fix problems with example code

Current code generates warnings and, in some cases, doesn't compile.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

test: Invalid local variables and it is in an infinite loop

Signed-off-by: shiqi <1454927420@qq.com>

rbd-nbd: only set size to new_size when NBD_SET_SIZE successfully.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

rgw_file: fix double unref on rgw_fh for rename

Skip unref after unlink to fix the problem.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

rgw: fix init_bucket_for_sync retcode

init_bucket_for_sync error retcode should be negative

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

rbd-mirror: move replayer admin socket hook to anonymous namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: refresh local images after acquiring leader role

The local image id set should be up-to-date when attempting to
determine which images need to be deleted.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: utilize the mirroring watcher to receive update notifications

Fixes: http://tracker.ceph.com/issues/15029
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: templatize Threads helper class for mock tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13953 from linuxbox2/wip-rmdir-check

rgw_file: implement reliable has-children check (unlink dir)

Merge pull request #13951 from smithfarm/wip-drop-obsolete-bt

tests: drop obsolete Perl scripts

Merge pull request #13944 from liupan1111/wip-fix-nbd-issues

rbd-nbd: don't ignore --read-only option in BLKROSET ioctl

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13893 from hrchu/rgw-admin-api-doc

doc: Update adminops.rst

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13688 from theanalyst/fix/rgw/http-client-dout-msg

rgw: http_client clarify the debug msg function call

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13985 from ebozag/master

doc: Fixes a typo.

Reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #13761 from zhangsw/cleanup-rgw-unlinkobj

rgw: correct the debug info when unlink instance failed.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file:  remove unused rgw_key variable

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>