ceph-objectstore-tool: remove rm-past-intervals op

The OSD doesn't rebuild this on demand anymore.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: do not clear same_interval_since and PastIntervals

We dropped the OSD recalculation code for this.  Import it.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: print pg_info_t to stdoutt

Helpful for debugging

Signed-off-by: Sage Weil <sage@redhat.com>

osd: print summary for forced backfill/recovery to debug

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop OSD friend, finally!

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop redundant is_active() check before scrub_sched()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make update_snap_mapper_bits public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make on_removal() public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make get_primary() public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add get_pool()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use pg_id.pool() for pg pool id

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make sched_scrub() public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: get_must_scrub()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PGBackend: remove scrub_supported()

Everybody supports scrub.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: change 'debug dump_missing'

- json
- move into PG class

No users in the qa suite.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move bits of delete_dir (pg removal) into PG

We don't want to expose things like SnapMapper, OSDriver.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: define public interface for heartbeat peers

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use accessors for pg resurrection

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: simplify start_recovery_ops return value meaning

Instead of complex check in caller, return simple bool indicating whether
find_unfound() should be called with a rctx.

There is one remaining condition that can probably be simplified:

  if (!recovering.empty() ||
      work_in_progress || recovery_ops_active > 0 || deferred_backfill)
    return !work_in_progress && have_unfound();

but we will leave it for now.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: simplify unfound recovery case; rename method

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop unneeded write_if_dirty

Nothing in start_recovery_ops or stuck_on_unfound dirties state.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move a few assertions, flag change into start_recovery_ops()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move some more split machinery into PG

and move other split pieces into the public interface.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move write_if_dirty inside handle_{loaded,create}

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: get_need_up_thru(), get_same_interval_since()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: bit less verbose while loading pgs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move more recovery logic into PG

I suspect we eventually want to move the create_context and
dispatch_context into OSDService (if it isn't there already) and move
even more of this logic into PG.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move some osd logic into handle_pg_trim

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: hide peering_queue from OSD

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove build_past_intervals_parallel

This was added in the dumpling era to handle the case where we have lots
of PGs with stale or missing past_intervals.  These days, we update
PastIntervals as we inject new makes (via advance_map), when we
learn about new PGs via notify with get the PastIntervals at the same time,
and during PGCreate handling we have a separate path in
build_initial_pg_history.  This code is obsolete.

It also reaches deep into the PG class where it shouldn't!

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use PG::is_deleting()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move load_pgs cruft into read_state()

This only happens during initial load; move it there.

Also, since update_store_with_options

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make read_state, peek_map_epoch public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: simplify/streamline upgrade process

There's no reason for this to be exposed to the OSD.  Just do it directly
in read_state().

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: drop bufferlist passed from peek_map_epoch to read_state/read_info

This is no longer used or needed.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make init() public

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: on_shutdown -> shutdown in public interface

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add get_num_ref()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add get_pg_stats() to public interface

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add dump_pgstate_history() to public interface

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make get_watchers public, hide locking

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop legacy infos object cleanup

This won't be there post-luminous

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: make osr public

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use is_deleting()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use pg->pg_id const member, not get_pgid()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: reorder some members

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: protect get_last_peering_reset

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: protect get_role()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: cleanup and move ref counting public bits

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: protect pg state accessors

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add is_forced_recovery_or_backfill()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: use Listener instead of PG interface to check undersized

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: collection more of public at top of class

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: expose static methods for ceph-objectstore-tool

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move C_PG_FinishRecovery into header

Less code than delcaring it in 2 places (by making it a friend).

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: move C_PG_ActivateCommitted into header

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: clean up set_force_{recovery,backfill} interface

- update state under pg lock
- clean up PG interface
- log only when we adjust the state

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: start grouping public interface at top of class

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: is_deleting()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: clean up intrusive_ptr helpers

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: protect everything that doesn't have to be public

This lets us see what the *actual* public interface is that we are using.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: get_acting()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add set_last_scrub_stamp

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add get_history() accessor

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: inline lock_suspend_timeout

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: add get_last_user_version()

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use PG::get_pgid() instead of info.pgid

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18134 from liewegas/wip-21660

src/messages/MOSDMap: reencode OSDMap for older clients

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17974 from jcsp/wip-doc-mgr-iptables

doc: update firewall doc to mention ceph-mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

src/messages/MOSDMap: reencode OSDMap for older clients

We explicitly select which missing bits trigger a reencode.  We
already had jewel and earlier covered, but kraken includes all of
the previously mentioned bits but not SERVER_LUMINOUS.  This
prevents kraken clients from decoding luminous maps.

Fixes: http://tracker.ceph.com/issues/21660
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18070 from liewegas/wip-21613

osd: fix replica/backfill target handling of REJECT

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #17939 from mkogan1/wip-rgw-21196

rgw: update the usage read iterator in truncated scenario

Merge pull request #18028 from liewegas/wip-pick-ifname

common/pick_address: add {public,cluster}_network_interface option

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #18091 from vasilyMellanox/gid_idx_fix

msg/async/rdma: uninitialized variable fix

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18092 from tchaikov/wip-cmake-cleanup

cmake: do not link against unused or duplicated libraries

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18094 from xiexingguo/wip-tracker-21618

qa/standalone/scrub/osd-scrub-repair.sh: add extents flag into object_info_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17907 from amitkumar50/cov-rgw-10

rgw: Ignoring the returned error

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18106 from kmroz/wip-keystone-revocator-thread-assert

rgw: stop/join TokenCache revoke thread only if started.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #18120 from liewegas/wip-bluefs-allocation

os/bluestore: make bluefs behave better near enospc

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18114 from dillaman/wip-21670

librbd: list_children should not attempt to refresh image

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18118 from liewegas/wip-21470

os/bluestore: fix another aio stall/deadlock

Reviewed-by: Varada Kari <varada.kari@gmail.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18105 from gregsfortytwo/wip-sigpipe-porting

msg: reimplement sigpipe blocking

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17710 from Songweibin/wip-add-pybind

pybind/rados: add support open_ioctx2 API

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18079 from liewegas/wip-21618

qa/standalone/scrub/osd-scrub-repair: no -y to diff

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #18088 from dillaman/wip-21629

osd: additional protection for out-of-bounds EC reads

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: add bluestore_bluefs_min_free

We need at least ~1GB free so we can write out new SSTs (which are 256MB
each).

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: crash on enospc

We were passing this error back to rocksdb, which would then crash (much)
later with a corrupted SST file!  No good.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use normal Context for async deferred_try_submit

I'm not quite sure why the FunctionContext did not ever execute on the
finisher thread (perhaps the [&] captured some state on the stack that it
shouldn't have?).  In any case, using a traditional Context here appears
to resolve the problem (of the async deferred_try_submit() never executing,
leading to a bluestore stall/deadlock).

Fixes: http://tracker.ceph.com/issues/21470
Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: wake kv thread when blocking on deferred_bytes

We need to wake the kv thread whenever setting deferred_aggressive to
ensure that txns with deferred io that have committed but haven't submitted
their deferred writes get submitted.  This aligns us with the other
users of deferred_aggressive (e.g., _osr_drain_all).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18099 from dillaman/wip-21628

librbd: journal should ignore -EILSEQ errors from compare-and-write

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

osd/PG: add explicit CANCEL MBackfillReserve type

This lets us distinguish between REJECT and CANCEL for clarity.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: separate event for RemoteReservationCanceled

Right now we transparently map a RemoteReservationRejected into a
*Canceled event because this what peers send over the wire.  Even
once new peers start sending and explicit CANCEL, old peers will
still do so, so we'll maintain this mapping for a while.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17959 from gaosibei/branch2

RGW: Multipart upload may double the quota

osd/PG: separate verb 'Reject' from passive 'rejected'

This reduces pg->reject_reservation() callsites from 2 to 1 and
makes the state transitions a bit more explicit.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18087 from liu-chunmei/fix_dpdk_build

build: fix dpdk build error

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #18115 from mogeb/rados-doc

doc: add osd_max_object_size in osd configuration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18072 from dragonylffly/wip-optimize-assert

assert(false)->ceph_abort()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #18073 from dragonylffly/wip-check-monitor-address-configuration

mon: check monitor address configuration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18082 from Liuchang0812/wip-21624

os/bluestore: make BlueStore opened by start_kv_only umountable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18083 from Liuchang0812/wip-21625

tool: ceph-kvstore-tool doesn't umount BlueStore properly

Reviewed-by: Sage Weil <sage@redhat.com>