bluestore: set next object as ghobject_t::get_max() when start.hobj.is_max()

Signed-off-by: Haomai Wang <haomai@xsky.com>
Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #11491 from xiexingguo/xxg-wip-fs-1014

os/filestore: fix potential result code overwriting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11482 from ifed01/wip-bluestore-gc-fixes2

os/bluestore: GC infra refactor, more UTs and GC range calculation fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11490 from xiexingguo/xxg-wip-bluestore-1014

os/bluestore: use iterator for erase() method directly

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: use iterator for erase() method directly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/blustore: fix compile warning

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: fix GC gc_end_offset miscalculation

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: isolate GC stuff to be able to cover it with UT

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11487 from batrick/cephfs-gid

cephfs: add gid to asok status

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11357 from batrick/i17518

mds: use parse_filesystem in parse_role to handle exceptions and reuse parsing code

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11380 from stiopaa1/mds_snapserver_addOverrideToVirt

mds/SnapServer.h: add override to virtual functs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11402 from stiopaa1/mds_fsmapRemoveUnneededUseCount

mds/FSMap.cc: remove unneeded use of count

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11435 from stiopaa1/mds_damageTableMoveSharePtr

mds/DamageTable.cc: move shared ptrs

Reviewed-by: John Spray <john.spray@redhat.com>

os/filestore: fix potential result code overwriting

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11484 from ceph/wip-move-ranges-tp

tracing/objectstore.tp: add missing move_ranges_... tp

tracing/objectstore.tp: add missing move_ranges_... tp

Signed-off-by: Sage Weil <sage@redhat.com>

cephfs: add gid to asok status

This is useful e.g. for building a database with a unique ID for each
entity.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #11468 from xiexingguo/xxg-wip-bluestore-1013

os/bluestore: traverse buffer_map in reverse order when spliting BufferSpace

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11296 from liewegas/wip-fixes

osd: drop stray debug message

Merge pull request #11237 from liewegas/wip-ec-move

os: move_ranges_destroy_src

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11426 from ceph/wip-rgw-17371

rgw: loses realm/period/zonegroup/zone data: period overwritten if somewhere in the cluster is still running Hammer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11391 from agraf/fix-aarch64-crc-v3

AArch64: Detect crc32 extension support from assembler

Reviewed-by: Yazen Ghannam <yazen.ghannam@linaro.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11451 from liewegas/wip-bluestore-reshard-fixes

os/bluestore: fix writes that span existing shard boundaries

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: force reshard if new extent spans shard boundary

A new write may create a new lextent that spans an
existing shard boundary.  If that happens, set a flag
so that we force a reshard when the onode is written.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11452 from liewegas/wip-bluestore-keys

os/bluestore: fix onode vs extent key suffix
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: add checking to catch unspecified compression algorithm

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: traverse buffer_map in reverse order

Which is more efficient.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11454 from zealoussnow/wip-1013

doc: fix dead link of "os-recommendations" in troubleshooting-osd

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/config_opts: comment nit

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: kill bluestore_shared_blob_hash_table_size_ratio

Which is depreciated by https://github.com/ceph/ceph/commit/7260166da20bfe31469498e55265b40a12f80652#diff-6519c151b74b5a9355a697be278a4382

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix tips when adding buffer to cache

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: fix dead link of "os-recommendations" in troubleshooting-osd

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #11439 from xiexingguo/xxg-wip-bluestore-1012

os/bluestore: avoid polluting shard info if need resharding

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix onode vs extent key suffix

I set the extent key suffix as 'x', thinking that was
not a valid hex character and would let us quickly
identify extent keys (vs onode keys, which end with an
encoded u64). But that doesn't work: the keys encode
integer values in raw form--not in hex
(pretty_binary_string just prints it out that way in
the debug log).

Fix by appending 'o' to the onode keys, so that we will
always have a trailing 'o' or 'x' and can use the last
char reliably to determine the type.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11269 from yehudasa/wip-multipart-uploads-cp

rgw: multipart upload copy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11403 from bassamtabbara/wip-cmake-allocator-fix

cmake: find gperftools package for tcmalloc_minimal too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11442 from wjwithagen/wip-wjw-freebsd-readme

README.FreeBSD: update to match the bimonthly FreeBSD status report

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11449 from dzafman/wip-scrub-prep

test/osd-scrub-repair.sh: Use test case specific object names to help…

Reviewed-by: Kefu Chai <kchai@redhat.com>

README.FreeBSD: update to match the bimonthly FreeBSD status report

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test/osd-scrub-repair.sh: Use test case specific object names to help with diagnostics

Signed-off-by: David Zafman <dzafman@redhat.com>

os/bluestore: avoid polluting shard info if need resharding

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: remove dead code from reshard()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11422 from xiexingguo/xxg-wip-bluestore-1011

os/bluestore: fix race condtion during blob spliting

Reviewed-by: Sage Weil <sage@redhat.com>

mds/DamageTable.cc: move shared ptrs

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #11394 from liewegas/wip-bluestore-shared-blob-intrusive

os/bluestore: use std::unordered_map for SharedBlob lookup

mds: use parse_filesystem in parse_role

This allows us to reuse code in parse_filesystem and avoid
get_filesystem which may fail if the fscid does not exist. This would
result in the program (mon) aborting due to the uncaught exception.

Fixes: http://tracker.ceph.com/issues/17518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: group filesystem access methods

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use reference to avoid copy

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: fully encapsulate filesystems map

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

cmake: find gperftools package for tcmalloc_minimal too

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

Merge pull request #11233 from zealoussnow/wip-0927

doc: fix start development cluster operation in index.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: use std::unordered_map for SharedBlob lookup

Many blobs aren't shared.  Save 8 bytes per SharedBlob by using a normal
unordered_map instead of instrusive::set.

More importantly, perhaps, it avoids us having to tune the intrusive
unordered_set size manually.  std::unordered_map does this automatically
for you, but the intrusive one does not.  And it's unclear how to
statically size it given that it's a per-collection structure and we have
no idea how many objects we'll have, how many blobs per object, and how
many objects will be cloned.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11183 from liewegas/wip-msgr-features

msg: make loopback Connection feature accurate all the time

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: improve ExtentMap::reshard() readability a little

As ep, sp and esp can be a little confusing.

Signed-off-by: xiexingguo <xie.xingguo@zte.com.cn>

Merge pull request #11427 from ifed01/wip-bluestore-fix-test

test/store_test: fix errors on the whole test suite run caused by the…

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: add sanity check to shard offset when updating

Signed-off-by: xiexingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix race conditon during blob spliting

Signed-off-by: xiexingguo <xie.xingguo@zte.com.cn>

test/store_test: fix errors on the whole test suite run caused by the lack of properites restore in some test cases

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: don't scan omap kvpair during fsck()

Because it does not check anything, and removing it can
speed up the process of fsck()

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: kiill dead member from bluestore_shared_blob_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw: avoid corruption when running old radosgw-admin on a newer rgw

Fixes:http://tracker.ceph.com/issues/17371
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11150 from renhwztetecs/renhw-wip-mds-server-open

mds/server: merge the snapshot request judgment

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11209 from david-z/wip-clear-dup-logic-mdsmonitor

mon: clear duplicated logic in MDSMonitor

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11358 from stiopaa1/mds_sessionmaph_unneededCount

mds/SessionMap.h: remove unneeded use of count

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11362 from batrick/i17531

mds: respawn using /proc/self/exe

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11359 from batrick/i17276

client: add pid to metadata

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: Fix missing master zone for default region conversion

Fixes:http://tracker.ceph.com/issues/17371
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: mark configuration as converted to avoid corruption when running older admins

Fixes: http://tracker.ceph.com/issues/17371
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #11395 from dillaman/wip-17549

librbd: ignore notify errors on missing image header

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11404 from bassamtabbara/wip-fix-yasm-system-check

cmake: Fix for cross compiling

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11356 from dillaman/wip-17528

librbd: update internals to use optional separate data pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11297 from stiopaa1/mds_sessionmap_avoidCopyingPlusConst

mds/SessionMap.cc: avoid copying and add const

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11283 from jcsp/wip-print-legacy-client-fscid

mds: include legacy client fsid in FSMap print

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

cmake: Fix for cross compiling

The check for yasm tool was calling uname -m which will not
work when cross compiling. Use CMAKE_SYSTEM_PROCESSOR instead.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

mds/FSMap.cc: remove unneeded use of count

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #11206 from dzafman/wip-16474

osd: Remove extra call to reg_next_scrub() during splits

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #11399 from theanalyst/wip-release-notes

scripts/release-notes: allow title guesses from gh tags & description update

Reviewed-by: Loic Dachary <ldachary@redhat.com>

build/ops: ceph-release-notes guess pr title based on gh tags

We add a switch --use-tags which helps guess the component of the pr
based on gh tags, still needs work, but provides better estimates when
looking at really large release notes where guessing each pr would be
too much work

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

scripts: add description to ceph-release-notes

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

librbd: include pool id in data block prefix if data pool enabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initialize the data pool IO context during image open state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: store data pool id with image header during creation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11355 from dillaman/wip-17424

rbd: add support for separate image data pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10731 from yehudasa/wip-rgw-sync-plugins

rgw: sync modules, metadata search

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: ignore notify errors on missing image header

The rename op on v1 images fails since the header no longer exists. In
the general case, the removal of the header object will also fail the
watcher which has its own recovery path.

Fixes: http://tracker.ceph.com/issues/17549
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11363 from xiexingguo/xxg-wip-fix-rbd-warn

cls_rbd: silence compiler warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11371 from stiopaa1/mds_cinode_removeCount

mds/CInode.h: remove unneeded use of count

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11383 from stiopaa1/mds_fsmap_passSharedPtrByConstRef

mds/FSMap: pass shared_ptr by const ref

rbd: include data pool name within info command

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: support overriding image data pool when creating images

Fixes: http://tracker.ceph.com/issues/17424
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11290 from trociny/wip-17017

rbd-mirror HA: move librbd::image_watcher::Notifier to librbd::object_watcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11260 from runsisi/wip-fix-mirror-image-disable

librbd: fix rollback if failed to disable mirroring for image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11301 from liewegas/wip-bluestore-buffered-write

os/bluestore: allow default to buffered write

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #11370 from ceph/wip-krbd-unmap-options

rbd: expose rbd unmap options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11255 from dzafman/wip-scrub-boundary

osd: fix scrub boundary to not include a SnapSet

Reviewed-by: Sage Weil <sage@redhat.com>

msg: make loopback Connection feature accurate all the time

In 626360aab05545ddacb0ac28e54a70e31fd5695d we made the
OSD cluster loopback connection CEPH_FEATURES_ALL, but
all other loopback connections got features == 0.  I
can't come up with any reason we wouldn't want those
connections to have accurate feature bits, so let's just
use CEPH_FEATURES_ALL for all of them.

While we're here, make the cflags argument required.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11377 from xiexingguo/xxg-wip-bluestore-1008

os/bluestore: drop inline_dirty from struct ExtentMap

Reviewed-by: Sage Weil <sage@redhat.com>

AArch64: Detect crc32 extension support from assembler

The used compiler may or may not be recent enough to recognize the
crc32 extended cpu type. However, it does not really have to know about
them either, since all we do is pass inline assembly instructions to
the assembler.

This patch moves the crc cpu extension detection from compiler based
to assembler based, so that we can build optimized code even when the
compiler does not know about the cpu type yet.

Fixes: http://tracker.ceph.com/issues/17516
Signed-off-by: Alexander Graf <agraf@suse.de>

os/bluestore: narrow scope of 'pos'

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11281 from jcsp/wip-17466

mds: handle bad standby_for_fscids in fsmap

Reviewed-by: Greg Farnum <gfarnum@redhat.com>