David Coles [Tue, 20 Oct 2015 02:42:18 +0000 (19:42 -0700)]
pybind: Don't encode str on Python 2
If you attempt to call encode on a non-ASCII string, then a UnicodeDecodeError
will be raised.
Since str on Python 2 is an 8-bit string, it's possible that it's already UTF-8
encoded. As such we should just pass it through to the C API unmodified.
On Python 3 or if the user explicitly uses unicode, then we'll encode it to
UTF-8 for them.
David Coles [Mon, 19 Oct 2015 21:07:15 +0000 (14:07 -0700)]
pybind: Use basestring as string type for Python 2
Python 2's built-in string type is an 8-bit string, thus does not specify any
particular encoding representation (str and bytes are synonymous).
On the other hand, Python 3 strings are always Unicode strings and thus require
some form of encoding before they can be passed to char* based C APIs.
Because we now explicitly encode and decode strings used in C APIs, functions
that return "strings" will now return a Unicode string. To allow these values
to be safely round-tripped in Python 2, we must allow use of both Unicode
and 8-bit strings as "string" arguments. In Python 3, the Unicode str type is
used instead.
8-bit strings will be assumed to be using Python's default ASCII encoding,
while Unicode strings can accept any encodable string value.
David Coles [Thu, 15 Oct 2015 21:58:53 +0000 (14:58 -0700)]
pybind: Add Python 3 support for rados and rbd modules
Python 3 explicitly distinguishes between strings (which can contain any Unicode
character) and 8-bit byte strings. This means that we must explicitly encode and
decode strings that use C-style char* string APIs.
Functions that take a true-strings now have their values encoded to UTF-8 before
being passed to the C API, while functions that accept binary data only accept
the bytes type. To help with this the `cstr` helper function has been introduced
as a way of easily a string into a C-string compatible form.
There is also a number of general Python 3 compatibility fixes:
- dict.iteritems() is replaced with just dict.items()
- xrange() is replaced with just range
- Iterators now use the __next__ magic method
- zip() and map() now return iterators, rather than lists
- print() is now a function
- contextlib.nexted() is replaced with multiple manager with-statement (from Python 2.7)
This also required updating of the unit-tests for these modules, mostly due to
explicitly distinguishing between strings and byte strings.
John Spray [Tue, 13 Oct 2015 19:38:58 +0000 (20:38 +0100)]
CMake: add vstart convenience targets
These are for developers like me, who want to
quickly build just the bits they need for
interactive vstart stuff, and don't wait for wait
for a full make.
John Spray [Tue, 13 Oct 2015 13:40:27 +0000 (14:40 +0100)]
CMake: make mon and osd depend on EC plugins
...because the services won't start if their
git version stamps don't match those of the
plugins. This is for the benefit of developers,
so that we can do "make ceph-mon" and get a
vstart-able result.
Sage Weil [Tue, 13 Oct 2015 12:37:40 +0000 (08:37 -0400)]
debian/control: python-setuptools is a build dependency
cd ./ceph-detect-init ; python setup.py build
Traceback (most recent call last):
File "setup.py", line 23, in <module>
from setuptools import setup
ImportError: No module named setuptools
Samuel Just [Tue, 13 Oct 2015 01:09:58 +0000 (18:09 -0700)]
ReplicatedPG::_rollback_to: handle block on full correctly
If we fail to promote the clone due to a full cache tier, we need
to block writes on the head just as if we were waiting on the
promote. Generally therefore, we block writes on head if we block
an op on a full cache tier. Add a new blocked set to support this.
Fixes: 13434 Signed-off-by: Samuel Just <sjust@redhat.com>
The prior code caused binary omap values to be discarded. This fixes
them to use the same model as the xattr iterator, and correctly return
binary data as python strings, eg:
'object_prefix': '\x15\x00\x00\x00rbd_data.449d2ae8944a'
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
Jason Dillaman [Fri, 9 Oct 2015 19:10:38 +0000 (15:10 -0400)]
cls: new force-promotion flag for class methods
Class methods that use the cls_get_request_origin should not be
proxied to a base tier -- otherwise the origin will reflect the
cache tier instead of the client.
Fixes: #13380 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Fixes: #12958
Head objects are mutable, so removing them can race with object removal
and a later recreation, so we might end up cleaning them up when we don't
need to.
The KeyServer class has a public method get_auth() that returns a boolean
value. This value is being checked here - fix the conditional so it triggers
when get_auth() returns false.
John Spray [Thu, 8 Oct 2015 12:04:04 +0000 (13:04 +0100)]
client: don't mark_down on command reply
I guess this was handy when issuing single commands
from the CLI, but it breaks things badly when
trying to issue commands from a client
that's also going to carry on and do client IO.
Only track read-after-write and write-after-write IO dependencies
via the associated write completions. All IO events after a write
completion are considered to be dependent and can be pruned down
to at most the number of concurrent IOs. This reduces the prep
time from a simple 'rbd bench-write' from over 4 hrs down to seconds.
Fixes: #13378, #13384
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
John Spray [Wed, 23 Sep 2015 11:58:46 +0000 (12:58 +0100)]
qa: avoid using sudo in fsstress
This test required root in order to copy its built
binary into /usr (presumably to avoid rebuilding it).
That's not really a good thing anyway because there's
no guarantee that a binary in that path is the binary
we wanted, so just run the thing straight out of /tmp. The
build is really quick anyway.
Signed-off-by: John Spray <john.spray@redhat.com> Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Sage Weil [Wed, 7 Oct 2015 15:49:01 +0000 (11:49 -0400)]
os/FileStore: kludge sloppy hammer temp objects into temp collection
When we are running with a mixed hammer cluster, hammer primaries
will generate temp object names that are sloppy. Make sure we still
put them into the temp collection.
Note that this isn't a problem on write because the primary (hammer)
OSD generated the transaction and explicitly specified a temp
collection; it's only transactions we do on our own with the sloppy
temp ghobject_t that trip over this.
Fixes: #13395 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log