Yingxin Cheng [Mon, 10 Feb 2020 09:00:31 +0000 (17:00 +0800)]
crimson/net: remove duplicated error codes and conditions
The duplicated error codes and conditions were originally introduced to
match connection errors with both system category (thrown by seastar)
and generic category (thrown by standard library). Since error_code
with system category can be matched by error_condition with generic
category (see std::errc and
system_error_category::default_error_condition(int)), our duplicated
counterparts are not needed actually.
Yingxin Cheng [Wed, 22 Jan 2020 04:31:26 +0000 (12:31 +0800)]
crimson: implement and adopt shard-local messenger
Remove the constraints to start messenger as a sharded service, and
remove foreign pointers from messenger interfaces. This simplifies users
to manage shard-local messenger as a normal object.
Yingxin Cheng [Sun, 19 Jan 2020 07:54:48 +0000 (15:54 +0800)]
test/crimson: add delay when stopping seastar
Seastar has bugs to have events undispatched during shutdown, which will
result in memory leak and thus fail LeakSanitizer.
Notably, there is a chance to lose the the destruction task of
foreign_ptr<lw_shared_ptr<conntrack::load_balancer>> submitted by
conntrack::~handle(). See
https://github.com/scylladb/seastar/blob/96ccbbe5262bed6d50a2392f6a77e84a490503b2/include/seastar/net/posix-stack.hh#L86-L89
Adopt FixedCPUServerSocket and don't move sockets across cores after
connected/accepted.
Implement the messenger to be managed in one CPU only, since we have
encapsulated the seastar listen-on-all requirement inside
FixedCPUServerSocket, and the requirements of a cross-core messenger has
not been defined yet.
The messenger interfaces can also be simplified, but will be in another
patch.
Yingxin Cheng [Sun, 19 Jan 2020 06:53:24 +0000 (14:53 +0800)]
crimson/net: implement FixedCPUServerSocket
Adopt the policy load_balancing_algorithm::fixed to instruct seastar
(posix-stack) to allocate connected sockets on the designated core, so
we are not able to move them later in our application, which is not
supported by seastar and result in undefined behaviors.
Seastar requires server_socket to accept/listen on all available cores.
We encapsulate the related implementations in the new
FixedCPUServerSocket, so we can have a simpler shard-local crimson
messenger.
Even though FixedCPUServerSocket is a sharded service, only the service
located on the shard where the socket is originally created will be
serving incoming connections. it's not allowed to perform i/o with that
connected socket on other cores.
Sage Weil [Tue, 4 Feb 2020 03:28:40 +0000 (21:28 -0600)]
Merge PR #33020 into master
* refs/pull/33020/head:
osdc/Objecter: inline pool full check
osdc/Objecter: remove duplicated pause check code
osdc/Objecter: only pause if respects_full()
osdc/Objecter: move respects_full() to op_target_t
Sage Weil [Tue, 4 Feb 2020 03:28:19 +0000 (21:28 -0600)]
Merge PR #32831 into master
* refs/pull/32831/head:
common, include: drop the copy{_in} from bufferlist entirely.
os/bluestore: switch copy_in() users to bufferlist::iterator.
osdc: switch users of bufferlist::copy{_in} to iterators.
osd: switch users of bufferlist::copy{_in} to iterators.
rgw: switch copy{_in} users to bufferlist::iterator.
ec: switch users of bufferlist::copy{_in} to iterators.
cls/queue: switch users of bufferlist::copy{_in} to iterators.
client: switch users of bufferlist::copy{_in} to iterators.
*: switch trivial users of bufferlist::copy{_in} to iterators.
test/bl: switch copy{_in} users to bufferlist::iterator.
common, include: kill the bl::last_p member.
common: encode for std::list<T> doesn't use bl::copy_in() anymore.
cephadm: fix error handling in `command_check_host()`
`find_program()` raises `ValueError` when the executable hasn't been
found. It means we need to catch `ValueError` exception in
`command_check_host()` and raise `Error` instead of `RuntimeError` since
only `Error` is caught at the end.
Typical failure:
```
INFO:cephadm:/usr/bin/ceph:stderr Error ENOENT: New host mon1 failed check: ['INFO:cephadm:podman|docker (/bin/podman) is present', 'INFO:cephadm:systemctl is present', 'Traceback (most recent call last):', ' File "<stdin>", line 2820, in <module>', ' File "<stdin>", line 2434, in command_check_host', ' File "<stdin>", line 796, in find_program', 'ValueError: lvcreate not found']
```
This allows for evaluation of more complex use cases where IgnorePublicACLs and
the like are set which need to be evaluated for GET/HEAD requests as well
This API returns whether the Bucket Policies/ACLs are public. There are a couple
of caveats:
- AWS currently returns PolicyNotFound error in case a bucket policy doesn't
exist, though a non existant bucket policy would mean the default ACLs apply
where the bucket is private, so error return here seems like an error
- the API spec mentions TRUE and FALSE as the response IsPublic element value,
however in practice both boto/aws clients and AWS S3 return/expect a lowercase
response.
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Conflicts:
src/rgw/rgw_rest_s3.h
merge conflict after zipper rework, dropped a spurious newline in rgw_rest_s3.h
after get_obj_op decl.
src/rgw/rgw_common.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h:
merge conflict after bucket replication merge, trivial conflicts
When playing with cephadm, at multiple times, I've reached the max
number of attempt in `is_available()`
Increasing the `retry_max` helps to avoid failure like following:
```
INFO:cephadm:mgr not available, waiting (1/5)...
INFO:cephadm:mgr not available, waiting (2/5)...
INFO:cephadm:mgr not available, waiting (3/5)...
INFO:cephadm:mgr not available, waiting (4/5)...
INFO:cephadm:mgr not available, waiting (5/5)...
ERROR: mgr not available after 5 tries
Sage Weil [Fri, 31 Jan 2020 14:35:26 +0000 (08:35 -0600)]
mgr/cephadm: prefix daemon ids with hostname
This is friendlier to a human operator since they can immediately see
where an instance is located, as with the legacy scheme, while still
keeping the unique random suffix. Use a . to separate so that we can
set per-host options.
* refs/pull/31633/head:
cephfs-shell: Instead of assert use stat for tests in rmdir
cephfs-shell: Add function for common rmdir test code
cephfs-shell: Add rmdir test for non empty directory
cephfs-shell: Add rmdir -p test for non empty directory
cephfs-shell: Add rmdir -p test for non existing dir
cephfs-shell: Add rmdir -p test to delete all dirs in given path
cephfs-shell: Add rmdir -p test for root directory with empty directories
cephfs-shell: Add rmdir test for valid file
cephfs-shell: Add rmdir test for invalid directory
cephfs-shell: Add rmdir test for valid directory
cephfs-shell: Fix rmdir '-p' issues
Reviewed-by: Rishabh Dave <ridave@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Fri, 31 Jan 2020 23:31:31 +0000 (17:31 -0600)]
Merge PR #32806 into master
* refs/pull/32806/head:
common/bl: fix accessibility of bptr's _off and _len fields.
common/bl: drop get_raw() from the public buffer::ptr interface.
common: drop sharing of buffer::raw outside bufferlist.
projects / ceph.git / log