rgw: asio: check the remote endpoint before processing requests

`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit caa653196856ecdf50519a9a33195d5c4e3372af)

 Conflicts:
src/rgw/rgw_asio_frontend.cc
conflicts due to optional-yield-ctx changes in master

Merge pull request #30213 from smithfarm/wip-41449-mimic

mimic: core: mon: C_AckMarkedDown has not handled the Callback Arguments

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30150 from neha-ojha/wip-40769-mimic

mimic: bluestore: common/options: Set concurrent bluestore rocksdb compactions to 2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30069 from smithfarm/wip-40124-mimic

mimic: qa/rgw: don't use ceph-ansible in s3a-hadoop suite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30133 from smithfarm/wip-40850-mimic

mimic: rgw/multisite: Don't allow certain radosgw-admin commands to run on non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29203 from smithfarm/wip-40320-mimic

mimic: tests: make: *** [hello_world_cpp] Error 127 in rados

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30219 from vumrao/wip-vumrao-bluefs-shared-alloc-with-log-level-change-mimic

mimic: os/bluestore: apply shared_alloc_size to shared device with log level change

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30260 from smithfarm/wip-41723-mimic

mimic: build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30355 from pdvian/wip-41765-mimic

mimic: build/ops: ceph.spec.in: reserve 2500MB per build job

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30672 from smithfarm/wip-37520-mimic

mimic: msg/async: do not trigger RESETSESSION from connect fault during connection phase

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30784 from smithfarm/wip-41918-mimic

mimic: core: osd: scrub error on big objects; make bluestore refuse to start on big objects

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29258 from smithfarm/wip-39741-mimic

mimic: rgw: swift object expiry fails when a bucket reshards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29483 from pdvian/wip-40761-mimic

mimic: rgw: Save an unnecessary copy of RGWEnv

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29500 from pdvian/wip-40847-mimic

mimic: rgw: Don't crash on copy when metadata directive not supplied

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30073 from smithfarm/wip-40517-mimic

mimic: rgw: perfcounters: add gc retire counter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30077 from smithfarm/wip-40599-mimic

mimic: rgw_file: fix readdir eof() calc--caller stop implies !eof and introduce fast S3 Unix stats (immutable)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29276 from smithfarm/wip-40215-mimic

mimic: rgw_file:  fix invalidation of top-level directories

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29984 from pdvian/wip-41110-mimic

mimic: rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30074 from smithfarm/wip-40539-mimic

mimic: cls/rgw: keep issuing bilog trim ops after reset

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30105 from smithfarm/wip-41120-mimic

mimic: rgw: permit rgw-admin to populate user info by access-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30130 from smithfarm/wip-40629-mimic

mimic: rgw: data/bilogs are trimmed when no peers are reading them

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30233 from smithfarm/wip-41129-mimic

mimic: qa: use hard_reset to reboot kclient

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30234 from smithfarm/wip-40444-mimic

mimic: mds: cleanup unneeded client_snap_caps when splitting snap inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30235 from smithfarm/wip-40844-mimic

mimic: mon/MDSMonitor: use stringstream instead of dout for mds repaired

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30236 from smithfarm/wip-40853-mimic

mimic: cephfs: test_volume_client: fix test_put_object_versioned()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30238 from smithfarm/wip-40896-mimic

mimic: cephfs: ceph_volume_client: convert string to bytes object

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30239 from smithfarm/wip-40899-mimic

mimic: mds: evict an unresponsive client only when another client wants its caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #30240 from smithfarm/wip-41466-mimic

mimic: cephfs: mount.ceph: properly handle -o strictatime

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30241 from smithfarm/wip-41487-mimic

mimic: cephfs: client: return -EIO when sync file which unsafe reqs have been dropped

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30417 from pdvian/wip-41852-mimic

mimic: mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30443 from pdvian/wip-41856-mimic

mimic: cephfs: client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #29609 from jtlayton/wip-40162-mimic

mimic: cephfs: client: fix bad error handling in _lookup_parent

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29751 from pdvian/wip-41088-mimic

mimic: qa: sleep briefly after resetting kclient

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29812 from pdvian/wip-41094-mimic

mimic: qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29833 from pdvian/wip-41097-mimic

mimic: cephfs: avoid map been inserted by mistake

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29915 from pdvian/wip-41100-mimic

mimic: cephfs: fix a memory leak

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29940 from pdvian/wip-41108-mimic

mimic: mds: delay exporting directory whose pin value exceeds max rank id

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30108 from smithfarm/wip-40442-mimic

mimic: cephfs: client: set snapdir's link count to 1

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30228 from smithfarm/wip-40841-mimic

mimic: cephfs: client: support the fallocate() when fuse version >= 2.9

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30678 from sobelek/wip-42048-mimic

mimic: ceph-volume: fix warnings raised by pytest

qa/rgw: update default port in perl workunits

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1643879218638915073d5310b859a94d10ffeac6)

qa/rgw: extra s3tests tasks use rgw endpoint configuration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 09e992ff01b4ce286540e1230a30df67103f5968)

qa/rgw: add dnsmasq back to s3a-hadoop

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8c8a706b6f4cb3b2a5b89aa1eb06b7e47909852d)

qa/rgw: remove ceph-ansible from s3a-hadoop suite

Fixes: http://tracker.ceph.com/issues/39706
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0fc2c8ecee2b6233292b9fd1325347fd0fdf9171)

Conflicts:
qa/tasks/s3a_hadoop.py
- trivial context difference

qa/rgw: use default ports (80 or 443) unless overridden

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3ff5f0c2406e40d1024d8152e0ac9400302a757a)

qa/rgw: rgw task can override --rgw-dns-name on the command line

the value for rgw_dns_name isn't known until a machine is assigned, so
it can't be set via 'override: conf:'. add a per-client config option
to the rgw task so it can add the endpoint's hostname and/or s3website
hostname on the radosgw command line

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 658e5932fb79e2d33b73363b6ce76ff299809e16)

qa/rgw: allow rgw client config to override port

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 921faebb723c11686bf790ca424c952a786f358a)

Merge pull request #30135 from smithfarm/wip-41126-mimic

mimic: rgw: swift: refrain from corrupting static large objects when using nginx as a GET cache

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30134 from smithfarm/wip-41103-mimic

mimic: rgw: mitigate bucket list with max-entries excessively high

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30140 from smithfarm/wip-41131-mimic

mimic: rgw: Fix bucket versioning vs. swift metadata bug.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30141 from smithfarm/wip-41625-mimic

mimic: rgw/rgw_op: Remove get_val from hotpath via legacy options

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #30154 from smithfarm/wip-41265-mimic

mimic: rgw: don't throw when accept errors are happening on frontend

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30156 from smithfarm/wip-41270-mimic

mimic: rgw: Fix rgw decompression log-print

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30157 from smithfarm/wip-41342-mimic

mimic: rgw_file: readdir: do not construct markers w/leading '/'

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #30159 from pdvian/wip-41628-mimic

mimic: rgw: RGWCoroutine::call(nullptr) sets retcode=0

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30165 from smithfarm/wip-41381-mimic

mimic: rgw: housekeeping of reset stats operation in radosgw-admin and cls back-end

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30166 from smithfarm/wip-41478-mimic

mimic: rgw: make dns hostnames matching case insensitive

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30167 from smithfarm/wip-41483-mimic

mimic: rgw: fix potential realm watch lost

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30168 from smithfarm/wip-41484-mimic

mimic: rgw: fix list bucket with delimiter wrongly skip some special keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30510 from pdvian/wip-41897-mimic

mimic: rgw: fix data sync start delay if remote haven't init data_log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30170 from smithfarm/wip-41494-mimic

mimic: radosgw-admin: bucket sync status not 'caught up' during full sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30171 from smithfarm/wip-41578-mimic

mimic: rgw: fix CreateBucket with BucketLocation parameter failed under default zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30172 from smithfarm/wip-41632-mimic

mimic: rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30173 from smithfarm/wip-41638-mimic

mimic: rgw: gc remove tag after all sub io finish

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30253 from smithfarm/wip-41708-mimic

mimic: rgw: fix cls_bucket_list_unordered() partial results

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30340 from smithfarm/wip-39014-mimic

mimic: rgw admin: disable stale instance delete in a multiste env

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30450 from smithfarm/wip-41847-mimic

mimic: rgw: increase beast parse buffer size to 64k

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30652 from pdvian/wip-41971-mimic

mimic: rgw: ldap auth: S3 auth failure should return InvalidAccessKeyId

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #30660 from smithfarm/wip-40626-mimic

mimic: rgw_file: advance_mtime() should consider namespace expiration

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #28125 from dillaman/wip-39429-mimic

mimic: rbd: librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29294 from dillaman/wip-40234-mimic

mimic: rbd/action: fix error getting positional argument

Merge pull request #29565 from pdvian/wip-40881-mimic

mimic: rbd: Reduce log level for cls/journal and cls/rbd expected errors

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29872 from trociny/wip-41284-mimic

mimic: rbd-mirror: don't overwrite status error returned by replay

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29873 from trociny/wip-41423-mimic

mimic: qa/workunits/rbd: stress test `rbd mirror pool status --verbose`

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30117 from smithfarm/wip-41438-mimic

mimic: rbd-mirror: ignore errors relating to parsing the cluster config file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30121 from pdvian/wip-41622-mimic

mimic: tools/rbd-ggate: close log before running postfork

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30176 from smithfarm/wip-39316-mimic

mimic: krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #30177 from smithfarm/wip-40380-mimic

mimic: rbd: librbd: avoid dereferencing an empty container during deep-copy

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30178 from smithfarm/wip-40500-mimic

mimic: rbd: use the ordered throttle for the export action

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30424 from smithfarm/wip-41543-mimic

mimic: test/librbd: set nbd timeout due to newer kernels defaulting it on

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30569 from pdvian/wip-41955-mimic

mimic: rbd/tests: avoid hexdump skip and length options in krbd test

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osd/PG: scrub error when objects are larger than osd_max_object_size

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 5928fe8ca08f9e307c0353bd81753cf263e33d90)

os/bluestore: refuse to mkfs or mount if osd_max_object_size >= MAX_OBJECT_SIZE

BlueStore has its own object size limit (2^32-1).  Make sure the cluster
limit is below that or refuse to mkfs or mount.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 377fdb1484e9ed34c3d46d2a9b99df926531c675)

Merge pull request #30718 from smithfarm/wip-epel-release-snafu-mimic

mimic: build/ops: backport miscellaneous install-deps.sh and ceph.spec.in fixes from master

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #29241 from smithfarm/wip-38206-mimic

mimic: core: osds allows to partially start more than N+2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29225 into mimic

* refs/pull/29225/head:
common/secret.c: fix key parsing when doing a remount

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rbd: use the ordered throttle for the export action

This allows exports to STDOUT to use multiple concurrent operations
and also fixes a potential race condition with concurrent callbacks
and file seeking.

Fixes: http://tracker.ceph.com/issues/40435
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 751bf6957020ad02b3a29b005b9792b9c6547f96)

Conflicts:
src/tools/rbd/action/Export.cc
- mimic has g_conf->get_val and int64_t "rbd_concurrent_management_ops"

Merge pull request #30291 from tchaikov/wip-mimic-41447

mimic: osd/PrimaryLogPG: Avoid accessing destroyed references in finish_degr…

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30447 from tchaikov/wip-mimic-37866

mimic: mgr/localpool: pg_num is an int arg to 'osd pool create'

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30481 from pdvian/wip-41863-mimic

mimic: mon: ensure prepare_failure() marks no_reply on op

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30485 from sobelek/wip-41922-mimic

mimic: mon: show pool id in pool ls command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30543 from smithfarm/wip-39317-mimic

mimic: mgr/dashboard: RGW proxy can't handle self-signed SSL certificates

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30275 from pdvian/wip-41704-mimic

 mimic: osd/PrimaryLogPG: update oi.size on write op implicitly truncating ob…

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph.spec.in: fix Cython package dependency for Fedora

Fedora distros do not have python3?-Cython packages, but they do have
python3-Cython ones. Fix the BuildRequires so that we only use the
python3_version_nodots based version string for RHEL.

Fixes: https://tracker.ceph.com/issues/42032
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 40c5a18f81c67c96f6a35e592ac3141b87c19701)

ceph.spec.in: s/pkgversion/version_nodots/

`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6d2f66846d1b84595cfdf8ac13a7ed4e1ae608ce)

rpm: make Python build deps somewhat less confusing

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 0b79741a770acb5cee6de2dd88022cd2c5d15ef5)

Conflicts:
ceph.spec.in

rpm: Install python2-Cython in python2 environment

do_cmake.sh tests for Cython module version in python2

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 6de7111e735683f2c73d4c013d9611fc02b68985)

Conflicts:
ceph.spec.in

install-deps: do not install if rpm already installed

there is chance that the system already have epel-release-latest-7.noarch.rpm
installed, in that case, install-deps.sh should not fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 45a4173bcfe15a9bf56f6aeb6e6c35386a00d1c5)

install-deps.sh: add EPEL repo for non-x86_64 archs as well

Fixes: https://tracker.ceph.com/issues/42023
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 49d740118f9a0ee6b6bb82954630c654946f1751)

install-deps.sh: extract munge_debian_control()

this function will uncomment necessary dependencies on demand.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4fd179654e3c37c837899372859959283479cee6)

Conflicts:
install-deps.sh