Skip to content

Commit

Permalink
Include UCX_PROTO_ENABLE in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
pentschev committed Jun 28, 2024
1 parent ae26bb8 commit 02bc2a5
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 13 deletions.
23 changes: 16 additions & 7 deletions docs/source/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,13 @@ UCX-Py redefines some of the UCX defaults for a variety of reasons, including be

Below is a list of the UCX-Py redefined default values, and what conditions are required for them to apply.

Apply to all UCX versions:

::

UCX_RNDV_THRESH=8192
UCX_RNDV_SCHEME=get_zcopy

Apply to UCX >= 1.12.0, older UCX versions rely on UCX defaults:

::

UCX_CUDA_COPY_MAX_REG_RATIO=1.0
UCX_MAX_RNDV_RAILS=1
UCX_PROTO_ENABLE=n

Please note that ``UCX_CUDA_COPY_MAX_REG_RATIO=1.0`` is only set provided at least one GPU is present with a BAR1 size smaller than its total memory (e.g., NVIDIA T4).

Expand All @@ -45,6 +39,21 @@ UCX Environment Variables in UCX-Py

In this section we go over a brief overview of some of the more relevant variables for current UCX-Py usage, along with some comments on their uses and limitations. To see a complete list of UCX environment variables, their descriptions and default values, please run the command-line tool ``ucx_info -f``.

UCP CONTEXT CONFIGURATION
~~~~~~~~~~~~~~~~~~~~~~~~~

Configuration variables applying to the UCP context.

UCX_PROTO_ENABLE
````````````````

Values: y, n

Enable the new protocol selection logic, also known as "protov2". Its default has been changed to ``y`` starting with UCX 1.16.0.

The new protocol solves various limitations from the original "protov1" including, for example, invalid choice of transport in systems with hybrid interconnectivity, such as a DGX-1 where only a subset of GPU pairs are interconnected via NVLink. On the other hand, it may still lack proper support on not be as well tested for lesser common use cases, such as CUDA async and managed memory.


DEBUG
~~~~~

Expand Down
8 changes: 2 additions & 6 deletions ucp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,7 @@
logger.info("Setting UCX_RNDV_FRAG_MEM_TYPE=cuda")
os.environ["UCX_RNDV_FRAG_MEM_TYPE"] = "cuda"

if (
pynvml is not None
and "UCX_CUDA_COPY_MAX_REG_RATIO" not in os.environ
and get_ucx_version() >= (1, 12, 0)
):
if pynvml is not None and "UCX_CUDA_COPY_MAX_REG_RATIO" not in os.environ:
try:
pynvml.nvmlInit()
device_count = pynvml.nvmlDeviceGetCount()
Expand Down Expand Up @@ -98,7 +94,7 @@ def _is_mig_device(handle):
):
pass

if "UCX_MAX_RNDV_RAILS" not in os.environ and get_ucx_version() >= (1, 12, 0):
if "UCX_MAX_RNDV_RAILS" not in os.environ:
logger.info("Setting UCX_MAX_RNDV_RAILS=1")
os.environ["UCX_MAX_RNDV_RAILS"] = "1"

Expand Down

0 comments on commit 02bc2a5

Please sign in to comment.