init-radosgw*: don't require rgw_socket_path to be defined

Fixes: #11159
Backport: hammer, firefly

Scripts required rgw_socket_path to exist in order to start radosgw.
This is not needed.

Reported-by: Dan Mick <dmick@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 6823bcdcd0ce72cd223e809291f46d82da76115c)

Merge pull request #3966 from theanalyst/wip-10698-backport

rgw: fail s3 POST auth if keystone not configured

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3934 from dachary/wip-10665-firefly

rgw: Swift API. Dump container's custom metadata.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3929 from dachary/wip-10475-firefly

rgw: Swift API. Support for X-Remove-Container-Meta-{key} header.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3938 from dachary/wip-10770-firefly

rgw: send appropriate op to cancel bucket index pending operation

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3961 from dachary/wip-10106-firefly

rgw: flush xml header on get acl request

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3958 from dachary/wip-10062-firefly

rgw: check for timestamp for s3 keystone auth

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Reviewed-by: Valery Tschopp <valery.tschopp@switch.ch>

Merge pull request #3942 from dachary/wip-10844-firefly

mon: MonCap: take EntityName instead when expanding profiles

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3936 from dachary/wip-10724-firefly

 backport ceph-disk man page to Firefly

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3940 from dachary/wip-8753-firefly

ReplicatedPG::on_change: clean up callbacks_for_degraded_object

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3827 from dachary/wip-10259-firefly

osd_types: op_queue_age_hist and fs_perf_stat should be in osd_stat_t::o...

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3962 from dachary/wip-10150-firefly

ReplicatedPG::scan_range: an object can disappear between the list and t...

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3948 from dachary/wip-9891-firefly

DBObjectMap: lock header_lock on sync()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3949 from dachary/wip-9915-firefly

osd: cache tiering: fix the atime logic of the eviction

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3944 from dachary/wip-9193-firefly

messages/MWatchNotify: include an error code in the message

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3943 from dachary/wip-8011-firefly

ReplicatedPG: fail a non-blocking flush if the object is being scrubbed

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3941 from dachary/wip-10817-firefly

WorkQueue: make wait timeout on empty queue configurable

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3937 from dachary/wip-10762-firefly

mon: ignore osd failures from before up_from

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3933 from dachary/wip-10617-firefly

osd: do not ignore deleted pgs on startup

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3932 from dachary/wip-10546-firefly

mon: Monitor: fix timecheck rounds period

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3931 from dachary/wip-10512-firefly

osd: requeue blocked op before flush it was blocked on

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3930 from dachary/wip-10497-firefly

librados: Translate operation flags from C APIs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3955 from dachary/wip-10059-firefly

PG: always clear_primary_state

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3954 from dachary/wip-10014-firefly

ObjectStore: Don't use largest_data_off to calc data_align.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3947 from dachary/wip-9555-firefly

osd: check that source OSD is valid for MOSDRepScrub

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3964 from dachary/wip-10157-firefly

PGLog: include rollback_info_trimmed_to in (read|write)_log

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3960 from dachary/wip-6003-firefly

FileJournal: fix journalq population in do_read_entry()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3950 from dachary/wip-9985-firefly

Get the currently atime of the object in cache pool for eviction

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xinze Chi <xmdxcxz@gmail.com>

Merge pull request #3935 from dachary/wip-10723-firefly

Fix memory leak in python rados bindings

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3866 from ceph/wip-cot-firefly

Backport ceph-objectstore-tool changes to firefly

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3996 from dzafman/wip-10676

Fix ceph command manpage to match ceph -h

Reviewed-by: Xinxin Shu <xinxin.shu5040@gmail.com>

Merge pull request #3927 from dachary/wip-10351-firefly

mount.ceph: avoid spurious error message

Reviewed-by: Yan, Zheng <zyan@redhat.com>

doc: Minor fixes to ceph command manpage

Fixes: #10676
Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Fix ceph command manpage to match ceph -h (firefly)

Improve synopsis section
Fixes: #10676
Signed-off-by: David Zafman <dzafman@redhat.com>

rados.py: keep reference to python callbacks

If we don't keep a reference to these, the librados aio calls will
segfault since the python-level callbacks will have been garbage
collected. Passing them to aio_create_completion() does not take a
reference to them. Keep a reference in the python Completion object
associated with the request, since they need the same lifetime.

This fixes a regression from 60b019f69aa0e39d276c669698c92fc890599f50.

Fixes: #10775
Backport: dumpling, firefly, giant
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 36d37aadbbbece28d70e827511f1a473d851463d)

doc: Changes format style in ceph to improve readability as html.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 8b796173063ac9af8c21364521fc5ee23d901196)

rgw: fail s3 POST auth if keystone not configured

Fixes: #10698
This fixes issue introduced in 8b3dfc9472022ea45ad24e02e0aa21dfdad798f8,
where if user does not exist, we try keystone authentication. However,
if keystone is not configured we justt fall through without failing.
This would have failed later on due to bucket permissions, unless bucket
had a public write permissions.

Backports: Firefly
Reported-by: Valery Tschopp <valery.tschopp@switch.ch>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Conflicts:
src/rgw/rgw_rest_s3.cc

osd: check that source OSD is valid for MOSDRepScrub

Make sure the message we got from the peer OSD is valid.  Specifically,
this avoids a race like this:

- A marks down B
- B sends MOSDRepScrub
- A accepts connection from B as new
- A replies to scrub
- B crashes because msgr seq 1 < expected seq 1000+

See #8880 for the most recent fix for requests.

Fixes: #9555
Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 847e5e102522d651aa9687a54aaafcebf3afc596)

Conflicts:
    src/osd/OSD.cc: require functions first argument is now a reference

osd: pass Message* to most require_* helpers

These do nothing but op->get_req(); pass the Message* explicitly so that
non-OpRequest callers can use them.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 707a111d53efb09b3471dd3788b86d2bfee4e96f)

Conflicts:
src/osd/OSD.cc
src/osd/OSD.h
        significantly changes had to be made but in a simple way and
stays within the scope of the original commit

PGLog: include rollback_info_trimmed_to in (read|write)_log

Fixes: #10157
Backport: firefly, giant
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1fe8b846641486cc294fe7e1d2450132c38d2dba)

Conflicts:
src/osd/PGLog.cc
        in the context coll_t::META_COLL was replaced with META_COLL

FileJournal: fix journalq population in do_read_entry()

Fixes: 6003
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit bae1f3eaa09c4747b8bfc6fb5dc673aa6989b695)

Conflicts:
src/os/FileJournal.cc
        because reinterpret_cast was added near two hunks after firefly

osd/ReplicatedPG: fix compile error

From 1fef4c3d541cba360738437420ebfa2447d5802e.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4a9ad7dc2da6f4fa6a64235776a3f1d2799aef60)

ReplicatedPG::scan_range: an object can disappear between the list and the attr get

The first item in the range is often last_backfill, upon which writes
can be occuring.  It's trimmed off on the primary side anyway.

Fixes: 10150
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit dce6f288ad541fe7f0ef8374301cd712dd3bfa39)

rgw: flush xml header on get acl request

Fixes: #10106
Backport: firefly, giant

dump_start() updates the formatter with the appropriate prefix, however,
we never flushed the formatter.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit eb45f861343162e018968b8c56693a8c6f5b2cab)

rgw: check for timestamp for s3 keystone auth

This commit ensures that we check for timestamp of s3 request is within
acceptable grace time of radosgw
Addresses some failures in #10062
Fixes: #10062
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
(cherry picked from commit 4b35ae067fef9f97b886afe112d662c61c564365)

PG: always clear_primary_state on new interval, but only clear pg temp if not primary

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit f692bfe076b8ddb679c6d1a6ea78cc47f0876326)

PG: always clear_primary_state when leaving Primary

Otherwise, entries from the log collection process might leak into the next
epoch, where we might end up choosing a different authoritative log.  In this
case, it resulted in us not rolling back to log entries on one of the replicas
prior to trying to recover from an affected object due to the peer_missing not
being cleared.

Fixes: #10059
Backport: giant, firefly, dumpling
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit c87bde64dfccb5d6ee2877cc74c66fc064b1bcd7)

ObjectStore: Don't use largest_data_off to calc data_align.

If largest_data_off % CEPH_PAGE_SIZE != 0, the get_data_aligment return
a erro value. This make the FileJouranl::align_bl to memcopy much data.

Tested-by: Sage Weil <sage@redhat.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit a1aa70f2f21339feabfe9c1b3c9c9f97fbd53c9d)

Get the currently atime of the object in cache pool for eviction

Because if there are mutiple atime in agent_state for the same object, we should use the recently one.

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit 1fef4c3d541cba360738437420ebfa2447d5802e)

osd: cache tiering: fix the atime logic of the eviction

Reported-by: Xinze Chi <xmdxcxz@gmail.com>
Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit 622c5ac41707069ef8db92cb67c9185acf125d40)

DBObjectMap: lock header_lock on sync()

Otherwise, we can race with another thread updating state.seq
resulting in the old, smaller value getting persisted.  If there
is a crash at that time, we will reuse a sequence number, resulting
in an inconsistent node tree and bug #9891.

Fixes: 9891
Backport: giant, firefly, dumpling
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2b63dd25fc1c73fa42e52e9ea4ab5a45dd9422a0)

Conflicts:
src/os/DBObjectMap.cc
        because we have state.v = 1; instead of state.v = 2;

Merge pull request #3828 from dachary/wip-10425-firefly

librados: fix resources leakage in RadosClient::connect()

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #3826 from dachary/wip-10257-firefly

mon: PGMonitor: several stats output error fixes

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3824 from dachary/wip-10353-firefly

crush: set_choose_tries = 100 for erasure code rulesets

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3823 from dachary/wip-10787-firefly

mon/OSDMonitor: do not trust small values in osd epoch cache

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>

Merge pull request #3915 from dachary/wip-10080-firefly

SimpleMessenger: allow RESETSESSION whenever we forget an endpoint

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon: MonCap: take EntityName instead when expanding profiles

entity_name_t is tightly coupled to the messenger, while EntityName is
tied to auth.  When expanding profiles we want to tie the profile
expansion to the entity that was authenticated.  Otherwise we may incur
in weird behavior such as having caps validation failing because a given
client messenger inst does not match the auth entity it used.

e.g., running

ceph --name osd.0 config-key exists foo daemon-private/osd.X/foo

has entity_name_t 'client.12345' and EntityName 'osd.0'.  Using
entity_name_t during profile expansion would not allow the client access
to daemon-private/osd.X/foo (client.12345 != osd.X).

Fixes: #10844
Backport: firefly,giant

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
(cherry picked from commit 87544f68b88fb3dd17c519de3119a9ad9ab21dfb)

mon: Monitor: stash auth entity name in session

Backport: giant

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
(cherry picked from commit ca8e1efc0be9bffcfbdce5593526d257aa498062)

messages/MWatchNotify: include an error code in the message

Document the fields, while we are here.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7c7bf5fee7be397ef141b947f532a2a0b3567b42)

ReplicatedPG: fail a non-blocking flush if the object is being scrubbed

Fixes: #8011
Backport: firefly, giant
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 9b26de3f3653d38dcdfc5b97874089f19d2a59d7)

WorkQueue: make wait timeout on empty queue configurable

Fixes: 10817
Backport: giant, firefly, dumpling
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 5aa6f910843e98a05bfcabe6f29d612cf335edbf)

ReplicatedPG::on_change: clean up callbacks_for_degraded_object

Backport: dumpling, firefly, giant
Fixes: 8753
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit a5ecaa189d47a69466a6cd55fa4180e5c3092dc2)

rgw: send appropriate op to cancel bucket index pending operation

Fixes: #10770
Backport: firefly, giant

Reported-by: baijiaruo <baijiaruo@126.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit dfee96e3aebcaeef18c721ab73f0460eba69f1c7)

Conflicts:
src/rgw/rgw_rados.cc

mon: ignore osd failures from before up_from

If the failure was generated for an instance of the OSD prior to when
it came up, ignore it.

This probably causes a fair bit of unnecessary flapping in the wild...

Backport: giant, firefly
Fixes: #10762
Reported-by: Dan van der Ster <dan@vanderster.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 400ac237d35d0d1d53f240fea87e8483c0e2a7f5)

doc: Adds the updated man page for ceph-disk utility.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 016080d2c39919d73956e5e138ba5f079327aa44)

doc: Updates the man page for ceph-disk utility with some changes.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 8a48847cd46d4807ca1dbd13d3a561a693cdc877)

doc: Adds man page for ceph-disk utility.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 9a118d56d4a5b0a0456e9f092f5ae9293f7bf3f3)

doc: Removes ceph-deploy usage instances from ceph-disk man page.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 242dd1c0bbb728475a94f47740790b8a196d9804)

doc: Updates man page for ceph-disk utility.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit 7dcc85042b0c0a26e495f7574ce144d1083d15f8)

doc: Adds man page for ceph-disk utility.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
(cherry picked from commit a450cab2b8148cb8a9b043d629feccf89e5aabac)

Fix memory leak in python rados bindings

A circular reference was inadvertently created when using the
CFUNCTYPE binding for callbacks for the asynchronous i/o callbacks.
This commit refactors the usage of the callbacks such that the
Ioctx object does not have a class reference to the callbacks.

Fixes: #10723
Backport: giant, firefly, dumpling
Signed-off-by: Billy Olsen <billy.olsen@gmail.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 60b019f69aa0e39d276c669698c92fc890599f50)

rgw: Swift API. Support for X-Remove-Container-Meta-{key} header.

Fixes: #10475
Backport: hammer, firefly
Reported-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Dmytro Iurchenko <diurchenko@mirantis.com>
(cherry picked from commit f67bfa24fd6f69c2fcc0987eba8b6b426dd78320)

Conflicts:
src/rgw/rgw_rest.h
        trivial merge: prototype of an unrelated function changed
        s/is_object_op/!(s->object == NULL)/

rgw: Swift API. Dump container's custom metadata.

Fixes: #10665
Backport: hammer, firefly
Reported-by: Ahmad Faheem <ahmad.faheem@ril.com>
Signed-off-by: Dmytro Iurchenko <diurchenko@mirantis.com>
(cherry picked from commit 2f8d31e9b1c4b09506bd1b0dad635c6e820783b2)

Conflicts:
src/rgw/rgw_rest_swift.cc
        The first hunk conflicts because X-Storage-Policy was added
after firefly. The second hunk conflicts because the type of the
s->object data member changed after firefly but it is irrelevant
because the patch does not need to access s->object anymore.

osd: do not ignore deleted pgs on startup

These need to get instantiated so that we can complete the removal process.

Fixes: #10617
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 879fd0c192f5d3c6afd36c2df359806ea95827b8)

mon: Monitor: fix timecheck rounds period

Fixes: #10546
Backports: dumpling?,firefly,giant

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
(cherry picked from commit 2e749599ac6e1060cf553b521761a93fafbf65bb)

osd: requeue blocked op before flush it was blocked on

If we have request A (say, cache-flush) that blocks things, and then
request B that gets blocked on it, and we have an interval change, then we
need to requeue B first, then A, so that the resulting queue will keep
A before B and preserve the order.

This was observed on this firefly run:

  ubuntu@teuthology:/a/sage-2015-01-09_21:43:43-rados-firefly-distro-basic-multi/694675

Backport: giant, firefly
Fixes: #10512
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11bdfb4131ecac16d4a364d651c6cf5d1d28c702)

librados: Translate operation flags from C APIs

The operation flags in the public C API are a distinct enum
and need to be translated to Ceph OSD flags, like as happens in
the C++ API. It seems like the C enum and the C++ enum consciously
use the same values, so I reused the C++ translation function.

Signed-off-by: Matthew Richards <mattjrichards@gmail.com>
(cherry picked from commit 49d114f1fff90e5c0f206725a5eb82c0ba329376)

Conflicts:
src/librados/librados.cc
        comes from lttng tracepoints introduced after firefly

mount.ceph: avoid spurious error message

/etc/mtab in most modern distributions is a symbol link to
/proc/self/mounts.

Fixes: #10351
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bdd0e3c4bda97fe18487a58dd173a7dff752e1a2)

SimpleMessenger: allow RESETSESSION whenever we forget an endpoint

In the past (e229f8451d37913225c49481b2ce2896ca6788a2) we decided to disable
reset of lossless Pipes, because lossless peers resetting caused trouble and
they can't forget about each other. But they actually can: if mark_down()
is called.

I can't figure out how else we could forget about a remote endpoint, so I think
it's okay if we tell them we reset in order to clean up state. That's desirable
so that we don't get into strange situations with out-of-whack counters.

Fixes: #10080
Backport: giant, firefly, dumpling

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 8cd1fdd7a778eb84cb4d7161f73bc621cc394261)

0.80.9

osd: Get pgid ancestor from last_map when building past intervals

Fixed OSD::build_past_intervals_parallel() and PG::generate_past_intervals()

Fixes: #10430
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0c5b66da7a9ba516340d06d9e806beb9d1040d0e)

Conflicts:
src/osd/OSD.cc

osd: Pass oldest_map when generating past intervals

From load_pgs() the superblock hasn't been published yet
so we need to retrieve the value of oldest_map depending on the context.

Fixes: #10427
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7fb721c1ceb39b38ca2e653299bcf51e109411d7)

Conflicts:
src/osd/OSD.cc

osd: build_push_op() handle short reads so recipient doesn't crash

Fixes: #8121
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b9a04846d18e1c6621d7f2794ec1fae02875bed2)

ceph_objectstore_tool: fix check_output on python2.6

* backported the subprocess.check_output from python2.7

Fixes: #10756
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 15350a088d84bc6fc664f0d3f5d09b35f58b2144)

Conflicts:
src/test/ceph_objectstore_tool.py

ceph_objectstore_tool: For import get object_info_t available for each object

Add object_info_t to object_begin so we have at object create time
This will be useful for importing from multiple erasure coded exports

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 05d916ed12f361da48ef73953bcc0cef465fcc2a)

ceph_objectstore_tool: Handle import of pg which OSD has split

Fail import if import data doesn't include OSDMap and can't find it locally
See if local map can be read for import's epoch
Jump to current epoch like a split would if local map not present

Fixes: #9781
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit afda6e4f3b98cc1773fd014583dfb5e1f214a939)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: On import following a split skip objects no longer in pg

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 68b27e25a3729566c3a22c0b71f70f7f3aca29a3)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: Verify that object pool and pgid pool match

Also, earlier check for invalid --pgid with import op

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit da3be80497a1b1878ee5d2015f8840d202a83aa2)

ceph_objectstore_tool: Check for splits and fail import if there were splits

Add osdmap into metadata_section
On export put metadata_section before file data

Fixes: #9780
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 19fdeea8b67091ed044ebce25799d3237b4d734a)

ceph_objectstore_tool: Add special exit code for test infrastructure

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b1f12f09c0211b608178f5ca2e292ab1765ce620)

ceph_objectstore_tool: Check that pool exists before allowing import

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f24f646d870703b7b79563cdbc20920248be6142)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: Check cluster_fsid before allowing an import

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 196c8112dc9a6e8780b05d6c579cd7fbd5b07589)

ceph_objectstore_tool: Allow the metadata_section to be anywhere in the export

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 62dd912f1192b28700a15b02507a8c9efd710cb5)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: import-rados shouldn't import internal namespace objects

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f727d2eaf50b0351feb9f12dcd65d50fb6eff7e9)

ceph_objectstore_tool: Get g_ceph_context available to import-rados

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ddc4613ec71752e5dccbbfe6dc078b86f0f186a9)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: Fix import-rados skipping of snapshots

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fe936026ed87c9f95f3b7ad235b24c22e8de5f55)

ceph_objectstore_tool: read_fd() doesn't handle ^D from tty stdin, don't allow

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5cb692528e3ac0ebea3f1714b3ac43f69d176888)

ceph-objectstore-tool: Remove --pretty-format and use new --format options

Call new_formatter() with --format specified argument

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 22b71744bb0cb56434d5f6214ccea7d81f771860)

Conflicts:
src/tools/ceph_objectstore_tool.cc

ceph_objectstore_tool: Describe super_ver values

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0aeba0f216a54390288b5e3d6147deb31877f744)

ceph_objectstore_tool.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 78542f9a901c05e627b53b5306ea604be3bc82e8)