Samuel Just [Tue, 11 Jun 2019 00:31:20 +0000 (17:31 -0700)]
crimson/osd/: restructure client and peering operation handling
This patch:
* Breaks pg map handling out into a separate module (pg_map.*)
* Breaks osdmap waiting into a separate module
- Ops actually need to wait twice (potentially): once at
the osd and a second time at the pg. The same structure
is used for both.
* Op ordering is enforced via OrderedExclusivePipelineStages
defined in osd_operations.h.
- Peering and client ops each define a pipeline section
centered on the connection as well as one centered on the
pg.
Samuel Just [Wed, 22 May 2019 09:57:03 +0000 (02:57 -0700)]
osd/PeeringState: make PeeringCtx inherit from BufferedRecoveryMessages
This'll be convenient in crimson for compound messages, don't want
to share an actual PeeringCtx, but want to combine the messages
to send. Using inheritance here simply prevents another round of
changing all of the PeeringCtx member users.
Nathan Cutler [Wed, 5 Jun 2019 15:30:30 +0000 (17:30 +0200)]
build/ops: make "patch" build dependency explicit
This commit adds "patch" as an explicit build dependency. Before, the RPM
build was installing it in the build environment via some mechanism that I
don't entirely understand.
The "patch" command is used by the BuildBoost cmake module.
* refs/pull/28293/head:
mds: avoid passing null to SessionmMap::hit_session()
qa/cephfs: add test for blacklisted client eviction
mds: fix 'is session in blacklist' check in Server::apply_blacklist()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/28433/head:
cephfs-shell: Remove unnecessary empty line
cephfs-shell: Fix flake8 bare 'except' warnings
cephfs-shell: Remove unused variable 'e'
cephfs-shell: Fix flake8 use != to compare str literals warning
cephfs-shell: Fixe flake8 line too long error
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Matt Benjamin [Fri, 7 Jun 2019 14:20:01 +0000 (10:20 -0400)]
rgw_file: all directories are virtual with respect to contents
This change causes directory handles to always report an mtime of
"now." This is not an invalidate per se--it interacts with the
nfs implementation to produce that result when the implementation
updates its cached attributes. Hence, it can be modulated by timers
or other rules governing attribute caching at the upper layer.
Fixes: http://tracker.ceph.com/issues/40204 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Kefu Chai [Thu, 6 Jun 2019 02:58:35 +0000 (10:58 +0800)]
debian/: add ceph-crimson-osd package
this package is commented out using `# Crimson`. and it will be enabled
only if we are building "crimson" flavor builds.
this package depends on ceph-osd, because it'll basically be a drop-in
replacement of ceph-osd. and it shares all the dependencies of ceph-osd.
so let's just reuse ceph-osd package for now.
Kefu Chai [Thu, 6 Jun 2019 03:10:41 +0000 (11:10 +0800)]
install-deps.sh: populate WITH_SEASTAR in the same way as for_make_check
because `install-deps.sh` is executed using `source`, we have to pass
these options using env variables. but before this change, `WITH_SEASTAR` is used directly,
while `FOR_MAKE_CHECK` is checked and translated to a local variable
`for_make_check`. which, in my opinion, has better readability.
so, in this change, `WITH_SEASTAR` is translated to `with_seastar`
variable in `install-deps.sh`.
Matt Benjamin [Thu, 6 Jun 2019 20:54:43 +0000 (16:54 -0400)]
rgw_file: permit lookup_handle to lookup root_fh
Originally, root_fh was reachable from RGWLibFS::fh_cache, but
this was problematic as it has infinite lifetime and so is not
present in RGWLibFS::fs_lru.
To fix the above, RGWLibFS::root_fh came to live unhooked from the
handle caches which works because nfs-ganesha/the application
always has an instance handle. However, this silently breaks
READDIR event invalidation.
Fixes: https://tracker.ceph.com/issues/40196 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Sage Weil [Thu, 6 Jun 2019 17:14:49 +0000 (12:14 -0500)]
Merge PR #28363 into master
* refs/pull/28363/head:
osd/ClassHandler: use std::variant for storing func
osd/ClassHandler: no need to add `struct` before struct type
osd/ClassHandler: use ceph::mutex instead of Mutex
osd/ClassHandler: remove unused function
osd/ClassHandler: s/string/std::string/ in header
Reviewed-by: Samuel Just <sjust@redhat.com> Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Zhi Zhang [Thu, 6 Jun 2019 03:32:34 +0000 (11:32 +0800)]
mds: count purge queue items left in journal
MDS purge queue didn't have a perf counter to record how many items
still left in journal. Even when MDS restarted, there was no any hint to
know how many inodes haven't been really deleted from disks.
xie xingguo [Mon, 3 Jun 2019 08:43:25 +0000 (16:43 +0800)]
test: add parallel clean_pg_upmaps test
With parallel clean_pg_upmaps feature on, the total time cost
of the performance test which now can utilize up to 8 threads for
parallel upmap validating decreased from:
Note that by default the mon uses only 4 worker threads for
CPU intensive background work, you could further increase
the "mon_cpu_threads" option value if you decided the
time-consuming of clean_pg_upmaps still matters.
xie xingguo [Mon, 3 Jun 2019 08:10:22 +0000 (16:10 +0800)]
mon/OSDMonitor: do clean_pg_upmaps the parallel way if necessary
There could definitely be some certain cases we could reliably
skip this kind of checking, but there is no easy way to separate
those out.
However, this is clearly the general way to do the massive pg
upmap clean-up job more efficiently and hence should make sense
in all cases.
Jeff Layton [Mon, 3 Jun 2019 14:55:37 +0000 (10:55 -0400)]
msg: fix addr2 encoding for sockaddrs
Currently, the sockaddr sa_family field is being sent across the wire
without any sort of endianness conversion.
Redefine the new-style entity_addr_t encoding to encode the sa_family
field as little-endian. This will allow LE machines in the field to
keep limping along, but note that BE machines may break until they are
all fixed.
To do this, use the code that was initially written for BSD/Apple
machines to encode the different fields separately.
Fixes: http://tracker.ceph.com/issues/40114 Signed-off-by: Jeff Layton <jlayton@redhat.com>
Mike Christie [Tue, 14 May 2019 03:57:41 +0000 (22:57 -0500)]
rbd nbd: add support for netlink interface
This adds support for mapping and unmapping using the netlink
interface. If the user passes in --try-netlink during mapping then we
will try to use the netlink interface and if that is not supported use
the ioctl interface. For unmap we will always try netlink and if that is
not supported drop down to the ioctl.
Signed-off-by: Mike Christie <mchristi@redhat.com>
Mike Christie [Wed, 5 Jun 2019 04:42:40 +0000 (23:42 -0500)]
cmake and build: add netlink lib requirement detection
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.
Signed-off-by: Mike Christie <mchristi@redhat.com>
Mike Christie [Tue, 4 Jun 2019 18:32:09 +0000 (13:32 -0500)]
cmake: Move WITH_RBD option earlier
The next patch will add a WITH_RBD check to the LINUX check block so we
only check for the netlink libs when those 2 conditions are met, so this
moves the WIT_RBD option setup to before the LINUX check.
Signed-off-by: Mike Christie <mchristi@redhat.com>
projects / ceph.git / log