osd/ReplicatedPG: drop old hobject_t encoding compatibility

This was introduced in 0.47 ... we don't need it any more!

Signed-off-by: Sage Weil <sage@redhat.com>

hobject_t: conditional Comparator object

This one can go either bitwise or nibblewise, depending on what you give
pass to the STL container constructor.  E.g.,

 map<hobject_t, int, hobject_t::Comparator> mymap(hobject_t::Comparator(bitwise));

Signed-off-by: Sage Weil <sage@redhat.com>

use explicit ghobject_t comparators

Signed-off-by: Sage Weil <sage@redhat.com>

use explicit hobject_t comparators

Signed-off-by: Sage Weil <sage@redhat.com>

explicitly specify comparator for all ghobject_t maps and sets

We no longer have operator<

Signed-off-by: Sage Weil <sage@redhat.com>

explicitly specify comparator for all hobject_t maps and sets

This is necessary now that operator< is not defined.

Signed-off-by: Sage Weil <sage@redhat.com>

hobject_t: typed MIN_* helpers

Signed-off-by: Sage Weil <sage@redhat.com>

common/simple_cache: parameterize map<> comparator

Signed-off-by: Sage Weil <sage@redhat.com>

common/sharedptr_registry: parameterize map<> comparator

Signed-off-by: Sage Weil <sage@redhat.com>

common/shared_cache: parameterize the map<> comparator

Signed-off-by: Sage Weil <sage@redhat.com>

include/encoding: handle set<> with comparator

Signed-off-by: Sage Weil <sage@redhat.com>

include/types: operator<< for set with comparator

Signed-off-by: Sage Weil <sage@redhat.com>

include/types: map operator<< with comparator

Signed-off-by: Sage Weil <sage@redhat.com>

include/encoding: map<> encoders when comparator is specified

Signed-off-by: Sage Weil <sage@redhat.com>

common/hobject_t: remove comparison operators for [g]hobject_t

*All* comparisions, including everything in the OSD, needs to
conditionally use either bitwise or nibblewise sort order.  Force the
issue by using the explicit cmp methods.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: ignore CRUSH_ITEM_NONE when calculating interval features

Signed-off-by: Sage Weil <sage@redhat.com>

osd: allow sort order to randomly revert to nibblewise via debug config

Note that this option is misused and applied to some but not all OSDs they
will disagree on the sort order and get confused, potentially losing data.
It must be global to the cluster in order to be useful.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not trust last_backfill for purposes of missing objects if sort order is wrong

If the peer has the wrong sort order, we cannot trust lsst_backfill and we
do not know if the object is there.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: restart backfill on peers if last_backfill sort is off

If a peer has a last_backfill value but the wrong sort order, we need to
restart backfill at the beginning.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use accessor to set last_backfill

Always will in last_backfill and last_backfill_bitwise together.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: set acting, upacting features from osdmap, not peers

Set in start_peering_interval based on the osdmap, so that it is defined
deterministically on primary, replica, and stray.

Note that peer features are still a primary-only construct.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: clean up pg_info_t comments a bit

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: add last_backfill_bitwise flag to pg_info_t

We need to know what sort order last_backfill is based on.

Signed-off-by: Sage Weil <sage@redhat.com>

hobject_t: pad hash to 8 digits

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: call+test new collection_list variants

In particular, make SimpleListTest check both sorting methods and asssert
that the result is sorted.

Signed-off-by: Sage Weil <sage@redhat.com>

os/HashIndex: handle new bitwise sorting

Handle listing by hash with bitwise sorting.  This involves futzing with
the hex digit sort order by reversing the digit nibbles and so on.

Signed-off-by: Sage Weil <sage@redhat.com>

os/HashIndex: handle legacy nibblewise sort

We need to separate the bitwise and nibblewise implementations because
they diverge too much.  Start by converting the old code to a working
nibblewise sort (compensating for the new ghobject_t sort and new args).

Signed-off-by: Sage Weil <sage@redhat.com>

os/LFNIndex: pass through sort_bitwise

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: pass sort_bitwise down to CollectionIndex

Signed-off-by: Sage Weil <sage@redhat.com>

os/FileStore: use bitwise sort when we don't care (e.g., split)

Signed-off-by: Sage Weil <sage@redhat.com>

os/KeyValueStore: only support bitwise sort

Signed-off-by: Sage Weil <sage@redhat.com>

os/MemStore: only support bitwise sort order

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PGBackend: ask PG which sort order to use

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use bitwise collection_list sort when we don't care

Signed-off-by: Sage Weil <sage@redhat.com>

qa: fix misc collection_list callers

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objecstore-tool: use new api

Signed-off-by: Sage Weil <sage@redhat.com>

os: add sort type to ObjectStore interface (incomplete)

Note that this doesn't build, but it will be easier to review this way.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: add OSD_BITWISE_HOBJ_SORT feature

We will sort [g]hobject_t's bitwise (instead of nibblewise) if all
OSDs who have participated in peering support the feature.

Note that this means a latecomer PG who does a notify may not have
the feature while the acting set operates with the new sort order.
We will need to be careful about last_backfill in this case.. that patch
it coming.

Signed-off-by: Sage Weil <sage@redhat.com>

hobject_t: restore default comparators ... bitwise!

Note that this breaks compatibility for the moment.

Signed-off-by: Sage Weil <sage@redhat.com>

hobject_t: nibblewise and bitwise comparators

Signed-off-by: Sage Weil <sage@redhat.com>

os/LFNIndex: return vector from list_subdirs

We'll need this later when we sort the subdirs bitwise instead of
nibblewise.

Signed-off-by: Sage Weil <sage@redhat.com>

os/HashIndex: drop unused lower_bound arg

Signed-off-by: Sage Weil <sage@redhat.com>

os/memstore: fix omap_rmkey_range

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5331 from liewegas/wip-collection-list

os: clean up collection_list

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5271 from wonzhq/cache-rate

mon: add cache tier IO rate in the 'osd pool stats' command

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5507 from athanatos/wip-12436

blkdev.cc::get_device_by_uuid: do not leak cache

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5506 from athanatos/wip-12338

OSD: break connection->session->waiting message->connection cycle

Reviewed-by: Sage Weil <sage@redhat.com>

os/ObjectStore: drop get_ideal_list_min()

Unused.  Fix comment on get_ideal_list_max while we are here.

Signed-off-by: Sage Weil <sage@redhat.com>

os: drop snapid_t arg to collection_list

All this does is filter out objects with old snapids, but this can be
just as easily done in the one caller that specifies a non-zero value
(PGLS and PGNLS), which in fact already do that filtering.

Signed-off-by: Sage Weil <sage@redhat.com>

s/collection_list_impl/collection_list/

Signed-off-by: Sage Weil <sage@redhat.com>

Kill collection_list in CollectionIndex

Now we use CollectionIndex.collection_list_partial.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Kill Flat_index.

Too old, remove it entirely.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Kill collection_list

use collection_list_impl as well.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Kill collection_list_partial

Use collection_list_impl directly.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Kill collection_list_range

We can directly use collection_list_impl.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os/KeyValuestore:Refactor collection_list_range and collection_list_partial

Add collection_list_impl which abstract the common process
of collection list behavior.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os/CollectionIndex: use const ref for end

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5509 from ceph/wip-12536

common/hobject_t: correctly decode pre-infernalis hobject_t min

Reviewed-by: Samuel Just <sjust@redhat.com>

os/Filestore:Refactor collection_list_range and collection_list_partial

Add collection_list_impl which abstract the common process
of coleection list behavior.

Todo: Refactor Index::collection_list_partial as
well.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

os/Memstore:Refactor collection_list_range and collection_list_partial

Add collection_list_impl which abstract the common process
of collection list behavior.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #5508 from athanatos/wip-12410

OSDMonitor::preprocess_get_osdmap: send the last map as well

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5413 from guangyy/wip-12523

osd: copy the RecoveryCtx::handle for when creating RecoveryCtx instance from another one

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5488 from oritwas/wip-cmake-fix-rbd

cmake: add global lib to rbd

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5475 from cxwshawn/com-fix

rgw_user.h: modify interface comments.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5483 from oddomatik/patch-1

Fix location of --access=full in SWIFT user creation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5484 from cxwshawn/de-fix

ceph-dencoder: add RGWRegion, RGWZoneParams, RGWOLHInfo support.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5421 from ceph/wip-selinux-policy-no-user

SELinux support for ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Tested-by: Yuri Weinstein <yweinstei@redhat.com>

Merge pull request #5487 from ceph/wip-12619

mds: initialize InodeStoreBase::damage_flags

Reviewed-by: John Spray <john.spray@redhat.com>

cmake: add global lib to rbd

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

mds: initialize InodeStoreBase::damage_flags

Fixes: #12619
Signed-off-by: Yan, Zheng <zyan@redhat.com>

ceph-dencoder: add RGWRegion, RGWZoneParams, RGWOLHInfo support.

    since ceph-dencoder does not support the up-list structures, so
    I add these three supports to analyze data more quickly.

Signed-off-by: shawn chen <cxwshawn@gmail.com>

Fix location of --access=full in SWIFT user creation

--access=full was incorrectly placed in the output of the command, while it was supposed to have been a flag run with the command.

ceph.spec.in: Make SELinux opt-out, not opt-in

We should enable SELinux support by default on rpm-based systems that
build with SELinux support to get better QE coverage in future runs.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5478 from dachary/wip-test-rados-striper

tests: rados striper tests use 7116 instead of 7113

Reviewed-by: Sebastien Ponce <sebastien.ponce@cern.ch>

Merge pull request #5459 from cxwshawn/master

radosgw-admin: use cout instead of cerr to print help message.

Reviewed-by: Sage Weil <sage@redhat.com>

selinux: Add .gitignore file

The gitbuilders release script needs this. Otherwise, the ceph-release
build will fail because there were some untracked files.

Signed-off-by: Boris Ranto <branto@redhat.com>

ceph.spec.in: stop/start service on policy upgrade/removal

Signed-off-by: Boris Ranto <branto@redhat.com>

selinux: Update the SELinux policy rules

Few new denials were found while testing the policy. Updating the policy
rules to refelct that.

Signed-off-by: Boris Ranto <branto@redhat.com>

SELinux Makefile can't work in parallel

We need to force single-core compilation of SELinux policy files in the
sub-make target as SELinux Makefile does not work properly when run in
parallel mode.

Signed-off-by: Boris Ranto <branto@redhat.com>

ceph-disk: set selinux context

Ensure that SELinux context is properly set according to policy.We
need this as SELinux context is not preserved across mount points.

Signed-off-by: Milan Broz <mbroz@redhat.com>

selinux: Allow setuid and setgid to ceph-mon and ceph-osd

Signed-off-by: Boris Ranto <branto@redhat.com>

Update selinux policy (after local test).

Changes enerated with ceph-test package.

Signed-off-by: Milan Broz <mbroz@redhat.com>

Fix selinux context after intitial OSD mount.

The SELinux context is not preserved across file-systems, we need to
manually set the context of the root of the file-system in order to
preserve SELinux context inheritence rules.

Signed-off-by: Milan Broz <mbroz@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Add initial SELinux support

This patch modifies the build system and spec file to provide a support
for SELinux enforcing in an opt-in matter via ceph-selinux package.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #4860 from ceph/wip-nobjectiterator-crash

librados: fix empty NObjectIterator crash on comparisons

Reviewed-by: Sage Weil <sage@redhat.com>

librados: fix empty NObjectIterator crash on comparisons

This would segfault if you did a
NObjectIterator i;
i == ...

Fix comparisons by allowing pointer-wise comparisons
for iterators with impl==NULL, and asserting in
other methods that require a non-null ::impl

Signed-off-by: John Spray <john.spray@redhat.com>

tests: rados striper tests use 7116 instead of 7113

Change src/test/libradosstriper/rados-striper.sh monitor address to 7116
because src/test/mon/osd-crush.sh uses 7113 already.

Signed-off-by: Loic Dachary <ldachary@redaht.com>

Merge pull request #5474 from kylinstorage/wip-sparse_read_bugfix

osd/ReplicatedPG: sparse read should return (extent_map, data_bufferlist)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

osd/ReplicatedPG: sparse read should return (extent_map, data_bufferlist)

Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>

rgw_user.h: modify interface comments.

Signed-off-by: shawn chen <cxwshawn@gmail.com>

Merge remote-tracking branch 'origin/next'

Conflicts:
src/Makefile-client.am (link rbd to libglobal again)

Revert "rbd: remove dependency on non-ABI controlled CephContext"

This reverts commit fa78739933d7b9501116a12a4fbfd58b5ca99cdb.
Original approach broke krbd map/unmap functions and did not fully
address ABI issues.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5446 from ceph/wip-systemd

systemd support

Reviewed-by: Owen Synge <osynge@suse.com>

Merge pull request #5469 from theanalyst/doc/v0.94.3-release

doc: release notes for v0.94.3

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: release notes for v0.94.3

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

client/Makefile: ship InodeRef

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4753 from ceph/wip-client-inode-refs

client: hold reference for returned inode

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5466 from ceph/wip-12578

mds: open base inode's snaprealm after decoding snapblob

Reviewed-by: John Spray <john.spray@redhat.com>

mds: open base inode's snaprealm after decoding snapblob

base inode has no parent, so can we mark its snaprealm open
immediately. This change makes sure replica mdsdir's snaplream
is opened.

Fixes: #12578
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #5140 from tchaikov/wip-pg-ls-update-pending-rel-notes

doc: add changes to "pg ls*" commands to pending release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>