is the client side, allowing Windows hosts to consume rados, rbd and cephfs
resources.
+.. _building:
Building
--------
It may be called from a Linux environment, including Windows Subsystem for
Linux. MSYS2 and CygWin may also work but those weren't tested.
+This script currently supports Ubuntu 18.04 but it may be easily adapted to
+run on other Linux distributions, taking into account different package
+managers, package names or paths (e.g. mingw paths).
+
.. _win32_build.sh: win32_build.sh
The script accepts the following flags:
CLEAN_BUILD Clean the build directory.
SKIP_BUILD Run cmake without actually
performing the build.
+SKIP_TESTS Skip building Ceph tests.
+BUILD_ZIP Build a zip archive containing
+ the generated binaries.
+ZIP_DEST Where to put a zip containing $BUILD_DIR/ceph.zip
+ the generated binaries.
+STRIP_ZIPPED If set, the zip will contain
+ stripped binaries.
============ =============================== ===============================
+In order to build debug binaries as well as an archive containing stripped
+binaries that may be easily moved around, one may use the following:
+
+.. code:: bash
+
+ BUILD_ZIP=1 STRIP_ZIPPED=1 SKIP_TESTS=1 ./win32_build.sh
+
+In order to disable a flag, such as ``CLEAN_BUILD``, leave it undefined.
+
+Debug binaries can be quite large, the following parameters may be passed to
+``win32_build.sh`` to reduce the amount of debug information:
+
+.. code:: bash
+
+ CFLAGS="-g1" CXXFLAGS="-g1" CMAKE_BUILD_TYPE="Release"
+
Current status
--------------
-The rados and rbd binaries and libs compile successfully and can be used on
-Windows, successfully connecting to the cluster and consuming pools.
-
Ceph filesystems can be mounted using the ``ceph-dokan`` command, which
requires the Dokany package to be installed. Note that dokany is a well
maintained fork of the Dokan project, allowing filesystems to be implemented
Installing
----------
-Soon we're going to provide an MSI installed for Ceph. For now, unzip the
+The following project allows building an MSI installer that bundles ``ceph`` and
+the ``WNBD`` driver: https://github.com/cloudbase/ceph-windows-installer
+
+In order to manually install ``ceph``, start by unzipping the
binaries that you may have obtained by following the building_ step.
You may want to update the environment PATH variable, including the Ceph
that the configured directories exist (e.g. ``C:\ProgramData\ceph\out``).
Please use slashes ``/`` instead of backslashes ``\`` as path separators
-within ``ceph.conf`` for the time being. Also, don't forget to include a
-newline at the end of the file, Ceph will complain otherwise.
+within ``ceph.conf`` for the time being.
.. _windows_service:
Windows service
# Let's give it a hefty 100MB size.
rbd resize cirros-0.5.1-x86_64-disk.raw --size=100MB
- rbd-wnbd map cirros-0.5.1-x86_64-disk.raw
+ rbd device map cirros-0.5.1-x86_64-disk.raw
# Let's have a look at the mappings.
- rbd-wnbd list
+ rbd device list
Get-Disk
$mappingJson = rbd-wnbd show cirros-0.5.1-x86_64-disk.raw --format=json
.. code:: PowerShell
rbd create blank_image --size=1G
- rbd-wnbd map blank_image
+ rbd device map blank_image
$mappingJson = rbd-wnbd show blank_image --format=json
$mappingJson = $mappingJson | ConvertFrom-Json
Initialize-Disk -PassThru | `
New-Partition -AssignDriveLetter -UseMaximumSize | `
Format-Volume -Force -Confirm:$false
+
+Troubleshooting
+...............
+
+Wnbd
+~~~~
+
+For ``WNBD`` troubleshooting, please check this page: https://github.com/cloudbase/wnbd#troubleshooting
+
+Privileges
+~~~~~~~~~~
+
+Most ``rbd-wnbd`` and ``rbd device`` commands require privileged rights. Make
+sure to use an elevated PowerShell or CMD command prompt.
+
+Crash dumps
+~~~~~~~~~~~
+
+Userspace crash dumps can be placed at a configurable location and enabled for all
+applications or just predefined ones, as outlined here:
+https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps.
+
+Whenever a Windows application crashes, an event will be submitted to the ``Application``
+Windows Event Log, having Event ID 1000. The entry will also include the process id,
+the faulting module name and path as well as the exception code.
+
+Please note that in order to analyze crash dumps, the debug symbols are required.
+We're currently buidling Ceph using ``MinGW``, so by default ``DWARF`` symbols will
+be embedded in the binaries. ``windbg`` does not support such symbols but ``gdb``
+can be used.
+
+``gdb`` can debug running Windows processes but it cannot open Windows minidumps.
+The following ``gdb`` fork may be used until this functionality is merged upstream:
+https://github.com/ssbssa/gdb/releases. As an alternative, ``DWARF`` symbols
+can be converted using ``cv2pdb`` but be aware that this tool has limitted C++
+support.
+
+ceph tool
+~~~~~~~~~
+
+The ``ceph`` Python tool can't be used on Windows natively yet. With minor
+changes it may run, but the main issue is that Python doesn't currently allow
+using ``AF_UNIX`` on Windows: https://bugs.python.org/issue33408
+
+As an alternative, the ``ceph`` tool can be used through Windows Subsystem
+for Linux (WSL). For example, running Windows RBD daemons may be contacted by
+using:
+
+.. code:: bash
+
+ ceph daemon /mnt/c/ProgramData/ceph/out/ceph-client.admin.61436.1209215304.asok help
+
+IO counters
+~~~~~~~~~~~
+
+Along with the standard RBD perf counters, the ``libwnbd`` IO counters may be
+retrieved using:
+
+.. code:: PowerShell
+
+ rbd-wnbd stats $imageName
+
+At the same time, WNBD driver counters can be fetched using:
+
+.. code:: PowerShell
+
+ wnbd-client stats $mappingId
+
+Note that the ``wnbd-client`` mapping identifier will be the full RBD image spec
+(the ``device`` column of the ``rbd device list`` output).
#!/usr/bin/env bash
set -e
+set -o pipefail
SCRIPT_DIR="$(dirname "$BASH_SOURCE")"
SCRIPT_DIR="$(realpath "$SCRIPT_DIR")"
CEPH_DIR="${CEPH_DIR:-$SCRIPT_DIR}"
BUILD_DIR="${BUILD_DIR:-${CEPH_DIR}/build}"
DEPS_DIR="${DEPS_DIR:-$CEPH_DIR/build.deps}"
+ZIP_DEST="${ZIP_DEST:-$BUILD_DIR/ceph.zip}"
CLEAN_BUILD=${CLEAN_BUILD:-}
SKIP_BUILD=${SKIP_BUILD:-}
+# Usefull when packaging existing binaries.
+SKIP_CMAKE=${SKIP_CMAKE:-}
+SKIP_DLL_COPY=${SKIP_DLL_COPY:-}
+SKIP_TESTS=${SKIP_TESTS:-}
NUM_WORKERS=${NUM_WORKERS:-$num_vcpus}
DEV_BUILD=${DEV_BUILD:-}
+BUILD_ZIP=${BUILD_ZIP:-}
+# By default, we'll build release binaries with debug symbols attached.
+# If BUILD_ZIP and STRIP_ZIPPED are enabled, we'll strip the binaries
+# that we're going to archive.
+# Unfortunately we cannot use pdb symbols when cross compiling. cv2pdb
+# well as llvm rely on mspdb*.dll in order to support this proprietary format.
+STRIP_ZIPPED=${STRIP_ZIPPED:-}
# We'll have to be explicit here since auto-detecting doesn't work
# properly when cross compiling.
# -Wa,-mbig-obj does not help.
CMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE:-RelWithDebInfo}
+binDir="$BUILD_DIR/bin"
+strippedBinDir="$BUILD_DIR/bin_stripped"
depsSrcDir="$DEPS_DIR/src"
depsToolsetDir="$DEPS_DIR/mingw"
mkdir -p $BUILD_DIR
cd $BUILD_DIR
+if [[ -z $SKIP_CMAKE ]]; then
# We'll need to cross compile Boost.Python before enabling
# "WITH_MGR".
echo "Generating solution. Log: ${BUILD_DIR}/cmake.log"
-D WNBD_LIBRARIES="$wnbdLibDir/libwnbd.a" \
-G "$generatorUsed" \
$CEPH_DIR 2>&1 | tee "${BUILD_DIR}/cmake.log"
+fi # [[ -z $SKIP_CMAKE ]]
if [[ -z $SKIP_BUILD ]]; then
echo "Building using $NUM_WORKERS workers. Log: ${BUILD_DIR}/build.log"
make -j $NUM_WORKERS -C $target_subdir ${make_targets[$target_subdir]} 2>&1 | tee -a "${BUILD_DIR}/build.log"
done
fi
+
+if [[ -z $SKIP_DLL_COPY ]]; then
+ # Hopefully this path will be the same across distros.
+ # This depends on the thread library, we're currently using posix.
+ mingwVersion=`x86_64-w64-mingw32-c++-posix -dumpversion`
+ mingwTargetLibDir="/usr/lib/gcc/x86_64-w64-mingw32/$mingwVersion"
+ required_dlls=(
+ $zlibDir/zlib1.dll
+ $sslDir/bin/libcrypto-1_1-x64.dll
+ $sslDir/bin/libssl-1_1-x64.dll
+ $mingwTargetLibDir/libstdc++-6.dll
+ $mingwTargetLibDir/libgcc_s_seh-1.dll
+ /usr/x86_64-w64-mingw32/lib/libwinpthread-1.dll)
+ echo "Copying required dlls to $binDir."
+ cp ${required_dlls[@]} $binDir
+fi
+
+if [[ -n $BUILD_ZIP ]]; then
+ if [[ -n $STRIP_ZIPPED ]]; then
+ rm -rf $strippedBinDir
+ cp -r $binDir $strippedBinDir
+ echo "Stripping debug symbols from $strippedBinDir binaries."
+ x86_64-w64-mingw32-strip $strippedBinDir/*.exe \
+ $strippedBinDir/*.dll
+ fi
+ echo "Building zip archive $ZIP_DEST."
+ zip -r $ZIP_DEST $strippedBinDir
+fi
projects / ceph.git / commitdiff