Datacenter-Data-Delivery Network (D3N) uses high-speed storage such as NVMe flash or DRAM to cache
datasets on the access side.
Such caching allows big data jobs to use the compute and fast storage resources available on each
-Rados Gateway node at the edge.
+RADOS Gateway (RGW) node at the edge.
Many datacenters include low-cost, centralized storage repositories, called data lakes,
to store and share terabyte and petabyte-scale datasets.
than the bandwidth of a solid-state storage located at an edge node.
| D3N improves the performance of big-data jobs running in analysis clusters by speeding up recurring reads from the data lake.
-| The Rados Gateways act as cache servers for the back-end object store (OSDs), storing data locally for reuse.
+| The RADOS Gateways act as cache servers for the back-end object store (OSDs), storing data locally for reuse.
Architecture
============
returning any blocks which are cached locally, and forwarding missed requests to the block home location
(as determined by consistent hashing) in the next layer.
Cache misses are forwarded to successive logical caching layers until a miss at the top layer is resolved
-by a request to the data lake (Rados)
+by a request to the data lake (RADOS)
:sup:`*` currently only layer 1 cache has been upstreamed.
Running
-------
-To enable D3N on an existing RGWs the following configuration entries are required
-in each Rados Gateways ceph.conf client section, for example for ``[client.rgw.8000]``::
+To enable D3N on existing RGWs the following configuration entries are required
+in the :ref:`ceph-conf-database`, for example for ``client.rgw.8000``:
- [client.rgw.8000]
- rgw_d3n_l1_local_datacache_enabled = true
- rgw_d3n_l1_datacache_persistent_path = "/mnt/nvme0/rgw_datacache/client.rgw.8000/"
- rgw_d3n_l1_datacache_size = 10737418240
+.. prompt:: bash #
+
+ ceph config set client.rgw.8000 rgw_d3n_l1_local_datacache_enabled true
+ ceph config set client.rgw.8000 rgw_d3n_l1_datacache_persistent_path /mnt/nvme0/rgw_datacache/client.rgw.8000/
+ ceph config set client.rgw.8000 rgw_d3n_l1_datacache_size 10737418240
The above example assumes that the cache backing-store solid state device
is mounted at ``/mnt/nvme0`` and has 10 GB of free space available for the cache.
-The persistent path directory has to be created before starting the Gateway.
-(``mkdir -p /mnt/nvme0/rgw_datacache/client.rgw.8000/``)
+The directory must exist and be writable before starting the RGW daemon:
+
+.. prompt:: bash #
+
+ mkdir -p /mnt/nvme0/rgw_datacache/client.rgw.8000/
In containerized deployments the cache directory should be mounted as a volume::
(Reference: `Service Management - Mounting Files with Extra Container Arguments`_)
-If another Gateway is co-located on the same machine, configure it's persistent path to a discrete directory,
-for example in the case of ``[client.rgw.8001]`` configure
-``rgw_d3n_l1_datacache_persistent_path = "/mnt/nvme0/rgw_datacache/client.rgw.8001/"``
-in the ``[client.rgw.8001]`` ``ceph.conf`` client section.
+If another RADOS Gateway is co-located on the same host, configure its persistent
+path to a discrete directory, for example in the case of ``client.rgw.8001``:
+
+.. prompt:: bash #
+
+ ceph config set client.rgw.8001 rgw_d3n_l1_datacache_persistent_path /mnt/nvme0/rgw_datacache/client.rgw.8001/
-In a multiple co-located Gateways configuration consider assigning clients with different workloads
-to each Gateway without a balancer in order to avoid cached data duplication.
+In a multiple co-located RADOS Gateways configuration consider assigning clients with different workloads
+to each RADOS Gateway without a balancer in order to avoid cached data duplication.
-.. note:: Each time the Rados Gateway is restarted the content of the cache directory is purged.
+.. note:: Each time the RGW daemon is restarted the content of the cache directory is purged.
Logs
----