test: Start using GNU awk and fix archiving directory

awk uses some tests that the native FreeBSD awk does not support:
    like: BEGIN{print 0 < 90}

And TESTDIR is not set when calling ceph-helpers from smoke.sh
    So fix with keeping the archive in /tmp

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #23948 from libingyang-zte/master

doc: Fix Spelling Error of Radosgw

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23947 from tchaikov/wip-test-msgr-include

test/msgr: add missing #include

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23926 from Gangbiao/lgb-devel

doc: Document correction

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: Fix Spelling Error of Radosgw

Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>

Merge pull request #23850 from liewegas/wip-22544

osdc/Objecter: fix split vs reconnect race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #23557 from liewegas/wip-cmd-getval

common/cmdparse: cmd_getval_throws -> cmd_getval

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/msgr: add missing #include

it's a regression introduced by #23930

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #23851 into master

* refs/pull/23851/head:
mds: print is_laggy message once

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #23890 into master

* refs/pull/23890/head:
msg: lower verbosity on normal event

Reviewed-by: Sage Weil <sage@redhat.com>

mds: print is_laggy message once

Beacon::is_laggy gets called frequently which causes the debug log to get
spammed with messages.

Steps to reproduce:
- setup vstart cluster
- start IO from client (e.g. `cp -av /usr /mnt`)
- find connection to mon by the mds
  $ ss -tnp | grep ceph-mds
    ...
    ESTAB      0      0      127.0.0.1:58988              127.0.0.1:40928               users:(("ceph-mds",pid=13095,fd=37))
- block the connection using iptables on the MDS node:
  $ sudo iptables -I INPUT -p tcp --sport 58988 --dport 40928 -j DROP
- verify MDS log using:
  $ tail -f mds.a.log | grep beacon
    ...
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.672 7f2908a9d700  1 mds.beacon.a is_laggy 19.8876 > 15 since last acked beacon
    2018-09-05 19:21:05.784 7f2905a97700  5 mds.beacon.a Sending beacon up:active seq 114
    2018-09-05 19:21:05.784 7f2905a97700  1 -- 127.0.0.1:6813/2277776624 --> 127.0.0.1:40929/0 -- mdsbeacon(14162/a up:active seq 114 v7) v7 -- 0x7d1e7e2a80 con 0
    2018-09-05 19:21:05.784 7f2905a97700 20 mds.beacon.a sender thread waiting interval 4s

Fixes: http://tracker.ceph.com/issues/35250
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23645 from jcsp/wip-mgr-unknown-error

mgr: replace "Unknown error" string on always_on

Reviewed-by: Noah Watkins <nwatkins@redhat.com>

Merge pull request #23942 from trociny/wip-rados-rmpool

qa/workunits/rbd: replace usage of 'rados rmpool'

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: replace usage of 'rados rmpool'

This command was dropped.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge PR #23907 into master

* refs/pull/23907/head:
doc: Fix Spelling Error of Cephfs Shell

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #23930 from tchaikov/wip-include-cleanup

common,mon: add implicit `#include` headers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23909 from theanalyst/doc/releases/12.2.8

doc: release notes for 12.2.8 luminous

doc: release notes for 12.2.8 luminous

- Add a special note for upgrades from earlier releases
- Add a notable section mentioning the scrub and rgw options

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #23852 from tchaikov/wip-cmake-cleanup

cmake: link compressor plugins against lib the modern way

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23925 from cfanz/wip-fix-data-sync-report

rgw-admin: fix data sync report for master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23626 from Liuchang0812/rgw-put-cors-compatible

rgw: return err_malformed_xml when MaxAgeSeconds is an invalid integer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23835 from Songweibin/wip-import-34536

rbd: fix error import when the input is a pipe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23815 from shun-s/fix-acquire-lock-latency

librbd: fix possible unnecessary latency when requeue request

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23935 from LenzGr/bug-35686-missing-tooltip

mgr/dashboard: Added missing tooltip to settings icon

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23409 from ricardoasmarques/wip-role-management

mgr/dashboard: Role management from the UI

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23938 from dillaman/wip-qa-rbd-mkpool

qa/workunits/rbd: replace usage of 'rados mkpool'

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/workunits/rbd: replace usage of 'rados mkpool'

This command was dropped under commit 2c26fb0fe1, so use
'ceph osd pool create' instead.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Role management from the UI

Fixes: https://tracker.ceph.com/issues/24447
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #23937 from tchaikov/wip-crimson-test-denc

test/crimson: do not use unit.cc as the driver of unittest_seastar_denc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test/crimson: do not use unit.cc as the driver of unittest_seastar_denc

as unit.cc initializes the CephContext and all of Ceph's infratructure,
which is not necessary for the denc test. also, seastar's builtin allocator
only pre-allocates 32 << 20 bytes. it's enough for the denc test, but
not necessarily enough for create CephContext and its friends. an option is
to use seastar's app template to initialize the memory allocator properly.
another option is to avoid initializing CephContext in this test.

the latter is simpler.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Added missing tooltip to settings icon

Fixes: https://tracker.ceph.com/issues/35686
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #23797 from votdev/fix_user_notifications

mgr/dashboard: Fix notifications in user list and form

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mon/MonClient.h: include necessary header

MonClient references LockPolicy, so it should include
common/lock_policy.h .

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/TrackedOp.h: include spinlock.h as this header uses it

Signed-off-by: Kefu Chai <kchai@redhat.com>

common,crush,mds,messages,osd: add missing #include

they rely on dout.h to do this, but they should not.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23816 from cyx1231st/wip-seastar-msgr-fix

crimson/net: miscellaneous fixes to seastar-msgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Fix notifications in user list

The notification message title and text were inverted.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #23912 from tchaikov/wip-install-deps-openldap

install-deps: s/openldap-client/openldap24-client/

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

doc: Document correction
    "doc/ceph-volume/lvm/prepare.rst":  Remove duplicated words
    "doc/radosgw/cloud-sync-module.rst": Remove duplicated words

Signed-off-by: Gangbiao Liu <liugangbiao@cmss.chinamobile.com>

rgw-admin: fix data sync report for master zone

Signed-off-by: cfanz <songxinying@sensetime.com>

Merge pull request #23918 from dzafman/wip-fix-assert

osd: Change assert() to ceph_assert() missed in the transition

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23759 from cbodley/wip-rgw-curl-timeout

rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23751 from joke-lee/wip-copy-response

rgw: etag in rgw copy result response body rather in header

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23749 from joke-lee/wip-etag-copy-part

rgw: fix Etag error in multipart copy response

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23646 from ashitakasam/get_cors_err_code

rgw: the error code returned by rgw is different from amz s3 when getting cors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23590 from yaozongyou/fix-rgw-deadlock

rgw: fix deadlock on RGWIndexCompletionManager::stop

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: Change assert() to ceph_assert() missed in the transition

Caused by: c789fd1899f697a994a28e45a6f9262c6721d77d

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #23917 from Gangbiao/master

doc: Fix Spelling Error In File "ceph.rst"

msg: lower verbosity on normal event

Fixes: http://tracker.ceph.com/issues/35251
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23827 from trociny/wip-mock-valgrind

 test/librbd: fix valgrind warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: Fix Spelling Error In File "ceph.rst" and "rbd.rst"

Signed-off-by: Gangbiao Liu <liugangbiao@cmss.chinamobile.com>

Merge pull request #23898 from tchaikov/wip-crimson-net-cleanup

crimson/net: s/repeat/keep_doing/

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crimson/net: s/repeat/keep_doing/

repeat ~= while() {}
keep_doing ~= for (;;) {}

so it's simpler this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps: s/openldap-client/openldap24-client/

please refer:

https://svnweb.freebsd.org/ports/head/net/openldap24-client/
https://svnweb.freebsd.org/ports/head/net/openldap-client/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23842 from votdev/issue_34315

mgr/dashboard: Display RGW user/bucket quota max size in human readable form

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #23905 from idryomov/wip-cram-suite-repo

qa/tasks/cram: use suite_repo repository for all cram jobs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

crimson/net: only do zero-copy writes to out_stream

seastar doesn't support mixed buffered writes and zero-copy writes.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

crimson/net: init h.global_seq during handshake

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

crimson/net: remove unused and dup global_seq

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

crimson/net: use parameter tag in handle_connect_reply()

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

crimson/net: support multiple call to conn.close()

It is possible while closing a connectin during a msgr shutdown,
a second close() can be called in `read_tags_until_next_message()`

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

Merge pull request #23491 from p-na/per-osd-settings

mgr/dashboard: Add support for managing individual OSD settings in the backend

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: Fix Spelling Error of Cephfs Shell

Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>

Merge pull request #23735 from votdev/suggest_snapshot_name

mgr/dashboard: Auto-create a name for RBD image snapshots

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23627 from Devp00l/wip-hints-for-jest-failures

mgr/dashboard: Add hints to resolve unit test failures

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23903 from libingyang-zte/master

doc: Fix Spelling Error of Rados Operations

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: Fix Spelling Error of Rados Operations

Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>

Merge pull request #23900 from libingyang-zte/master

doc: Fix Spelling Error of Rados Operations

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23899 from luomuyao/cleanup

rgw: drop unused tmp in main()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: Fix Spelling Error of Rados Operations

Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>

rgw: drop unused tmp in main()

Signed-off-by: luomuyao <luo.muyao@zte.com.cn>

qa/tasks/cram: use suite_repo repository for all cram jobs

Currently git.ceph.com is hardcoded for all cram jobs.  Testing
modifications is a pain: one needs to push to either ceph/ceph.git or
ceph/ceph-ci.git (depending on where the ceph branch is at, triggering
unnecessary builds in the latter case) and wait for the mirror to sync.
Runs scheduled against branches in developer's forks fail.

Move away from git.ceph.com to allow mixing branches and repositories,
similar to workunits.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #23798 from liewegas/wip-pod-hostname

common/util: pass real hostname when running in kubernetes/rook container

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23322 from ricardoasmarques/wip-role-management-api

mgr/dashboard: Add REST API for role management

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add REST API for role management

Fixes: https://tracker.ceph.com/issues/25138
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add support for managing individual OSD settings (backend)

Add options to mark OSDs in/out/down/reweight/lost/remove/destroy/create

Fixes: http://tracker.ceph.com/issues/24270
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Auto-create a name for RBD image snapshots.

Signed-off-by: Volker Theile <vtheile@suse.com>

crimson/net: fix peer_addr sync in during handshake

Signed-off-by: Yingxin <yingxin.cheng@intel.com>

crimson/net: fix error category equivalence checks

consult base class for equivalence (which will match errors within
this category) before applying the extra error code mappings

Signed-off-by: Casey Bodley <cbodley@redhat.com>

crimson/net: add error::success = 0

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23891 from libingyang-zte/master

doc: Fix Spelling Error of Rados Operations

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/dashboard: Display RGW user/bucket quota max size in human readable form

Fixes: https://tracker.ceph.com/issues/34315
Signed-off-by: Volker Theile <vtheile@suse.com>

doc: Fix Spelling Error of Rados Operations

Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>

Merge pull request #23879 from dwahler/master

doc: add note about LVM volumes to ceph-deploy quick start

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: add note about LVM volumes to ceph-deploy quick start

This commit also fixes some incorrect indentation that was causing
previous steps in the same section to be incorrectly numbered.

Signed-off-by: David Wahler <dwahler@gmail.com>

cmake: link compressor plugins against lib the modern way

* always link against the target library not library paths
* import their include directories by linking against their target
  library, instead of doing so using target_include_directories()
  manually.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/cmdparse: cmd_getval_throws -> cmd_getval

Always throw.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: fix try/catch whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

osd: catch cmd_bad_get from do_command

Refactor to limit impact of try block.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: catch cmd_bad_get for asok hooks

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/DaemonServer: catch cmd_bad_get

Refactor handle_command slightly: establish the CommandContext, then call
_handle_command.  This keep the try block localized.

Signed-off-by: Sage Weil <sage@redhat.com>

common: catch cmd_bad_get

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSDaemon: catch cmd_bad_get

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_config: $host expands to ceph_get_short_hostname(), not hostname -s

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #23540 into master

* refs/pull/23540/head:
include/ceph_fs: rename old auid field
PendingReleaseNotes: note about auid support removal
radosgw-admin: remove -a --auth-uid arg
rgw: remove auid member from RGWUserInfo
auth: remove auid member from EntityAuth
osd: remove auid session member
mon: remove auid session member
doc/dev/cephx_protocol: drop auid reference
auth: remove auid args from handle_request and verify_authorizer
mon/OSDMonitor: remove 'osd pool {get,set} <name> auid ...'
mon/OSDMonitor: remove auid arg for 'osd lspools' and deprecate
osd/OSDCap: remove auid from grammar
osd/OSDCap: remove auid from is_capable() etc args
auth: clean up cap parse error messages
mon/AuthMonitor: raise health warning on invalid caps
mon/AuthMonitor: drop ancient auth inc encoding compat
messages/MPoolOp: drop auid member
osdc/Objecter: drop change_pool_auid
pybind/rados: drop auid arg to pool_create
pybind/rados: drop change_auid
rados: drop mkpool, rmpool commands
rados: remove 'chown' command
librados: deprecate calls that take auid
librados: mark all auid calls deprecated
mon/OSDMonitor: drop variable pool auid for prepare_new_pool
mon/OSDMonitor: remove pool auid change support
osdc/Objecter: do not pass auid to create_pool
ceph-authtool: remove auid options
qa/workunits/cephtool: remove auid tests

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #23739 into master

* refs/pull/23739/head:
mgr/telemetry: add crashdump info to report

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #23766 into master

* refs/pull/23766/head:
BlueStore: Fix concurrency conflicts

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: fix split vs reconnect race

If we are reconnecting to an OSD, we may not have a connection established
so this feature check is completely unreliable.  If there happens to be a
split, that could mean we don't resend an op that should have been resent.

Instead, we can infer the OSD has the new behavior if require_osd_release
is luminous or later.  If not, we can look at the per-osd features in the
OSDMap.

Fixes: http://tracker.ceph.com/issues/22544
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23841 from idryomov/wip-krbd-namespaces

krbd: support for images within namespaces

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23663 from xiexingguo/wip-incompat-async-fixes

osd: some recovery improvements and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>