rgw: url_decode values from X-Object-Manifest during GET on Swift DLO.

Fixes: #12728
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5470 from ceph/wip-newstore-sort

osd: change sort order for hobject_t

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #5424 from lebauce/fix_swift_user_command_line

doc: fix command line for swift user creation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5048 from SUSE/wip-objecter-comment-cleanup

Objecter.cc: fix name of _take_op_budget() function in comment

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5606 from liewegas/wip-scrub-errormsg

osd/ReplicatedPG: fix missing whitespace in scrub error message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5604 from oritwas/wip-cmake-fix-rbd

cmake: add DiffIterate.cc to librbd

Reviewed-by: Kefu Chai <kchai@redhat.com>

moved to use boost uuid implementation, based on commit 4fe89a7b14c97b2ed7f357132901beb2bdcec551

Signed-off-by: Rohan Mars <code@rohanmars.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #5433 from zqkkqz/master

common/Thread: pthread_attr_destroy(thread_attr) when done with it

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/ReplicatedPG: fix missing whitespace in scrub error message

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5538 from ceph/wip-selinux-improve-update

selinux: Relabel files if and only if the policy version changed

Reviewed-by: Milan Broz <mbroz@redhat.com>

ceph.spec.in: Require awk for selinux post script

Signed-off-by: Boris Ranto <branto@redhat.com>

selinux: Relabel files if and only if the policy version changed

Currently, the ceph files are being relabelled every time the package is
rebuilt. Fix this by checking the policy versions and relabel the files
only if the policy actually changed (different policy version was
detected).

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5596 from mbroz/wip-selinux-systemd

Update ceph-selinux rpm spec

Reviewed-by: Boris Ranto <branto@redhat.com>

cmake: add DiffIterate.cc to librbd

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #5600 from liewegas/wip-osd-bind

doc: ceph-osd binds to up to 4 ports, not 3

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: ceph-osd binds to up to 4 ports, not 3

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/next'

Merge pull request #5597 from dachary/wip-12718-restorecon

ceph-disk: only call restorecon when available

Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-disk: only call restorecon when available

9db80da12803d42bb676d67f37442c0c54d83448 added an unconditional call to
restorecon after mounting the filesystem. It fails when restorecon is
not available and must be made conditional.

http://tracker.ceph.com/issues/12718 Fixes: #12718

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Do not use hardcoded paths in spec post section for selinux commands.

Signed-off-by: Milan Broz <mbroz@redhat.com>

Add dependence to selinux-policy-base to fix installation in kickstart.

Signed-off-by: Milan Broz <mbroz@redhat.com>

Update rpm spec to properly restart systemd ceph.target.

If we are using systemd, ceph target should be used.
Also fail to restart service should not stop policy to load.

Signed-off-by: Milan Broz <mbroz@redhat.com>

Merge pull request #5595 from ceph/wip-12710

qa/fsstress.sh: fix 'cp not writing through dangling symlink'

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5594 from ceph/wip-12711

mds: properly set client incarnation

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa/fsstress.sh: fix 'cp not writing through dangling symlink'

On some test machines, /usr/lib/ltp/testcases/bin/fsstress is
dangling symlink. 'cp -f' is impotent in this case.

Fixes: #12710
Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: properly set client incarnation

Fixes: #12711
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #5593 from ceph/revert-4927-snapset-obc

Revert "osd/ReplicatedPG: snapset is not persisted"

Reviewed-by: Samuel Just <sjust@redhat.com>

Revert "osd/ReplicatedPG: snapset is not persisted"

Merge pull request #5577 from oritwas/wip-next-12363

rgw: we should not overide Swift sent content type

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5558 from s09816/rbd-fix

rbd:improve the error handle of rbd,check the return value.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/OSDMap: test_flag returns bool

Otherwise comparing the result to a bool will promote the bool to
an int and fail the comparison.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: restart listing this PG if sort order changes

If the cluster sort order changes mid-way through our listing, our
cursor within this pg is meaningless and we need to restart at the
beginning of the PG.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: fix missing set sort order on [N]PGLS

This is the only thing that cares about the missing set sort order,
so only adjust it here as needed.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: add pg_missing_t::resort() method

Set/change sort order as needed.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: make pg_missing_t sort order dynamic

Signed-off-by: Sage Weil <sage@redhat.com>

osd: refuse to boot if SORTBITWISE not set but backend cannot sort nibblewise

This prevenents OSDs with new backends from starting if the
cluster is still using the legacy sort order.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5586 from ceph/wip-jni-loader

wip-jni-loader

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #5588 from zhouyuan/isal_yasm_fix

configure: Fix checking for yasm compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

configure: Fix checking for yasm compability

Fix typo when checking yasm

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

java: add libcommon to deps

Why do indirect dependencies seem to work randomly:

  undefined symbol: _ZN4ceph3log3Log12create_entryEii

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

java: search for JNI bits in common dirs

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #5572 from xinxinsh/wip-rados-bench-error

fix print error of rados bench

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd:improve the error handle of rbd,check the return value.

Signed-off-by: s09816 <shi.lu@h3c.com>

Merge pull request #5443 from ceph/wip-wrlock

cleanup: remove all traces of rados 'lock' operations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge remote-tracking branch 'origin/next'

Merge pull request #5537 from s09816/master

rbd:modify the log of purging snaps so that it is more appropriate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5560 from solesoul1127/master

rbd:Check the dest image name, if it is empty string, return error and give a message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

fix print error of rados bench

Total time run:         12.279167
Total writes made:      92
Write size:             4194304
Bandwidth (MB/sec):     30
Stddev Bandwidth:       23.4
Max bandwidth (MB/sec): 64
Min bandwidth (MB/sec): 2
Average IOPS:           7
Stddev IOPS:            6
Max IOPS:               32767
Min IOPS:               -1537890352
Average Latency:        2.12
Stddev Latency:         1.35
Max latency:            6.05
Min latency:            0.501

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

Merge pull request #5569 from ceph/wip-unused-var

client: fix unused var warning

rbd:modify the log of purging snaps so that it is more appropriate.

Signed-off-by: s09816 <shi.lu@h3c.com>

Merge pull request #4744 from ceph/wip-11625

librbd: diff_iterate should issue concurrent ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5555 from dachary/wip-mon-test-timeouts

tests: be more generous with mon tests timeouts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5576 from liewegas/wip-dencoder

simplify handling for objects w/ nondeterministic encoding

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: we should not overide Swift sent content type

Fixes: #12363
backport: hammer

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

test/encoding/check-generated: test sorted json dumps for nondeterministic objects

Nondeterministic objects dump nondeterministically (usually due
to unordered_map or _set).  Compare their sorted output.  This
is a weaker test but is better than nothing.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "osd/HitSet: make subclasses dump deterministically"

This reverts commit 34c048b3cb2b16658e692ac5110115612031510d.

Revert "mon/PGMap: dump osd_epochs in deterministic order"

This reverts commit 3df6438cd2c9a77a06b7a8d011bafbed00f1fc5e.

Merge pull request #4864 from stevecapperlinaro/for-ceph/journal-page-size-fix

FileJournal: Remove CEPH_PAGE_SIZE assumptions

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5100 from stevecapperlinaro/for-ceph/tweak-buffer-append-size

Common: Do not use CEPH_PAGE_SIZE when appending buffers in Ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5339 from xinxinsh/wip-use-throttle

use throttle framework to throttle objectstore ops/bytes

Reviewed-by: Li Wang <liwang@ubuntukylin.com>
Reviewed-by: Sage Weil <sage@redhat.com>

crypto: fix unbalanced ceph::crypto::init/ceph::crypto:shutdown

we may create a CephContext without calling common_init_finish(), then
delete the CephContext. In this case, ceph::crypto:init() is not called,
so CephContext::~CephContext() should not call ceph::crypto::shutdown().

Fixes: #12598
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5573 from SUSE/wip-restart-on-rpm-removal-only

ceph.spec.in: test %preun argument is zero for removal-only operations

ceph.spec.in: test %preun argument is zero for removal-only operations

The %preun section now contains logic for disabling and stopping all the
Ceph systemd units when the ceph package is removed. However, there is no
conditional around it, so the units are disabled and stopped on RPM upgrade
as well as removal.

http://tracker.ceph.com/issues/12685 Fixes: #12685

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>

Merge pull request #5562 from varadakari/wip-kvstore-statfs

Adding statfs api to KeyValueDB

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5565 from varadakari/wip-kvstore-makefile

Remove redundant line from Makefile

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: do not test timeout mon add

Strictly speaking it's not a feature, but an expected hahaviour. After
discussion it does not seem useful to have that test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>

client: fix unused var warning

Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'origin/next'

Merge pull request #5564 from wonzhq/test-inode

mds: initialize 'divergent' to false when comparing inode_t

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5553 from majianpeng/mds

mds (fix to flags_to_mode and cleanup in handle_client_open)

Reviewed-by: John Spray <john.spray@redhat.com>

mds/Server: clean up code for handle_client_open.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/ceph_fs: set mode default is -1 for func ceph_flags_to_mode.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Remove redundant line from Makefile

Signed-off-by: Varada Kari <varada.kari@sandisk.com>

mds: initialize 'divergent' to false when comparing inode_t

'divergent' is not set in the case when all the fields of 'inode_t' are
equal.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>

Adding statfs api to KeyValueDB

If any backend supports and implements statfs will extract the
stats from backend to show correct utilization in the status.

Signed-off-by: Varada Kari <varada.kari@sandisk.com>

rbd:Check the dest image name, if it is empty string, refuse to execute and give a message

Signed-off-by: solesoul1127 <chen.yehua@h3c.com>

Merge pull request #5544 from ceph/wip-12664-next

librbd: prevent race condition between resize requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: be more generous with mon tests timeouts

Change the timeouts of the TEST_mon_add_to_single_mon tests to be 120
seconds. They have value to guard against blocking forever. The exact
timing does not matter as long as the operation completes.

When a timeout is too short, it will create false negatives when running
on slow machines. It could be argued that being too generous with
timeout in general may hide problems. But not in this specific case.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5550 from schuzhoy/schuzhoy-patch-1

doc: Correcting the default number of copies.

Reviewed-by: Travis Rhoden <trhoden@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5072 from zhouyuan/doc_rgw_rados_handlers

doc: update rgw configuration on multiple rgw rados handlers feature

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5495 from tchaikov/wip-12620

doc/rados/operations/add-or-rm-mons: simplify the steps to add a mon

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5528 from Be-El/master

Correct handling of supplementary groups in ceph-fuse (Bug 12617)

doc: update rgw configuration on multiple rgw rados handlers feature

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Doc: Correcting the default number of copies.

According to the "quick ceph deploy" page, the default number of replicas in the Ceph configuration file is 3

Signed-off-by: Sasha Chuzhoy <sashac88@hotmail.com>

Merge pull request #5047 from rzarzynski/wip-12108

rgw: rework X-Trans-Id header to be conform with Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5449 from yangruifeng/wip-rgw-bugfix1

rgw:segmentation fault when rgw_gc_max_objs > HASH_PRIME

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5447 from yangruifeng/master

rgw:the arguments 'domain' should not be assigned when return false

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5543 from ceph/wip-cmake-nlist

CMake: add missing librados nlist test

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5547 from ritz303/wip-doc-11082

doc: Removed reference to RAID-4

Reviewed-by: John Wilkins <jowilkin@redhat.com>

doc: Removed reference to RAID-4

Fixes: #11082
Signed-off-by: ritz303 <ritz_303@yahoo.com>

Merge pull request #5546 from ritz303/wip-doc-11897

doc: Clarify how 'ceph auth caps' works.

Reviewed-by: John Wilkins <jowilkin@redhat.com>

CMake: cut down unnecessary linkage on rados tests

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: add missing librados nlist test

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #5354 from ceph/wip-pgls-filter-librados

librados: partially expose PGLS filter interface

Reviewed-by: David Zafman <dzafman@redhat.com>

doc/rados/operations/add-or-rm-mons: simplify the steps to add a mon

this change removes the step to "ceph mon add" before starting a new
monitor. because the existing leader will start an election at seeing
the MMonJoin message sent by the new joiner, after the quorum is
archieved, the monmap will be updated with the new monitor.
so, "ceph mon add" is not necessary to add a new monitor.
moreover, this command will be blocked until a new quorum is formed,
and the proposed monmap is accepted. but in case of adding a monitor
to a single monitor cluster, the leader will wait until at least two
of the monitors reply to it. apparently, this does not happen unless
the new monitor starts. so from the user's point of view, this
command hangs until timesout, if he/she does not start the mon.b
beforehand. but this is an expected behaviour.

so, to avoid this confusion and simplify the steps to add a new
monitor. we'd better simply remove this "ceph mon add" step.

Fixes: #12620
Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: prevent race condition between resize requests

It was possible that the same resize request could be sent twice
if a completed resize op started a newly created resize op while
it was also being concurrently started by another thread.

Fixes: #12664
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5535 from ceph/wip-gentest

mon: Improve PGMap::generate_test_instances(), build map per incremental

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5457 from ceph/wip-cmake-vstart

Make vstart work with cmake out of tree builds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'upstream/master'

test: add case for pgls filter arg in librados

Signed-off-by: John Spray <john.spray@redhat.com>