Omniverse Client Library Changes

2.23.0

• HUB-206: Build client library for Mac

• OM-73588: Fix trace propagation

• OM-74155: Fixes an issue on Linux where some old XFS filesystems of ftype=0 would ignore recursive directories when moving files.

• HUB-442: Update build tools to Visual Studio 2019 & C++17

• OM-75253: Fix leak of connection object when auth service returns ‘token expired’

• HUB-453: Correctly emit ‘created’ event when a file is copied on Nucleus.

• OM-76648: Fixed crash at shutdown in Python callback thread.

• OM-75668: Fix omniClientGetLocalFile returning a path with a leading ‘/’ on Windows

• OM-52351: Use configured chuck size for download also

• HUB-469: Added omniClientBypassListCache

• HUB-455: Add more eOmniClientConnectionStatus enums:

  • eOmniClientConnectionStatus_Connecting: Attempting to connect

  • eOmniClientConnectionStatus_Connected: Successfully connected

  • eOmniClientConnectionStatus_ConnectError: Connection error while trying to connect

  • eOmniClientConnectionStatus_Disconnected: Disconnected

  • eOmniClientConnectionStatus_SignedOut: omniClientSignOut called

  • eOmniClientConnectionStatus_NoUsername: No username was provided

  • eOmniClientConnectionStatus_AuthAbort: Application returned an abort code in the callback provided to omniClientRegisterAuthCallback

  • eOmniClientConnectionStatus_AuthCancelled: User clicked “Cancel” or the application called omniClientAuthenticationCancel

  • eOmniClientConnectionStatus_AuthError: Internal error while trying to authenticate

  • eOmniClientConnectionStatus_AuthFailed: Authentication failed

  • eOmniClientConnectionStatus_ServerIncompatible: The server is not compatible with this version of the client library

Note: The changes below are disabled unless you are running the (currently experimental) Hub

• HUB-437: Add OMNICLIENT_USE_HUB environment variable.

• HUB-339: Add OMNICLIENT_HUB_EXE environment variable to launch hub on demand.

• HUB-385: Bypass Nucleus cache when Hub is enabled.

• HUB-450: Fixed Hub cache not being filled when using extReadAsset.

• HUB-456: Update to capnproto 0.10.3

• HUB-400: Add retry logic when two clients access the same file at the same time.

Unit test changes/fixes:

• HUB-440: Hub unit tests use their own local folders

• HUB-446: Fix flaky “test_stat_deleted” test

• HUB-376: Enable hub tests in CI/CD/CT

2.22.0

• Adding connectivity to the OmniHub

• OM-70781: Read aliases from omniverse.toml - also add python bindings for set_alias

• OM-73388: Add support for “file://server/share” URLs

• OM-23286: Fix “Cannot stat file in root”

• OM-73588: CreateObject and LiveRead/UpdateObject omnitrace instrumentation

• HUB-206: Added support for an initial build for Mac.

Note: The changes below are disabled until Nucleus 115 is released

• HUB-317: Support soft-delete in the client-library

  • The C omniClientDelete function and the Python “delete..” functions now mark the files/folders as “deleted” instead of removing them from the Nucleus server entirely.

  • If a file has been deleted, and then a new file or file version is created at the same path by any operation, the restored file will have all the checkpoints of the previously deleted file.

    • To be backwards compatible, the copy/move/write commands on a deleted file will succeed even if the “ErrorIfExists” flag is set.

  • If a folder has been deleted, and a new file or folder is created in it, the folder will be restored.

  • All reading operations on a deleted file or folder will fail with ERROR_NOT_FOUND, unless the optional parameter include_deleted is set.

    • The existing list/stat functions in the C and Python API are backwards compatible - they will not list deleted files.

    • In the C API new functions omniClientStat2, omniClientStatSubscribe2, omniClientList2 and omniClientListSubscribe2 have been introduced which support listing all files, including deleted files.

    • In the Python API the existing list and stat functions have an additional optional parameter include_deleted to allow listing all files, including deleted files.

  • The return result of the C-API omniClientGetServerInfo and the Python get_server_info.. functions now contains a flag “undelete_enabled” to indicate if the server supports soft-delete, obliterate and undelete.

• HUB-94: Added omniClientObliterate, plus matching event and list events, which allows deleting files permanently from the server.

  • Obliterate on a folder will fail unless the folder is empty.

  • Obliterate is only supported for the Nucleus provider.

• HUB-91: Added omniClientUndelete, which allows restoring a deleted file or folder.

  • Undelete on a folder will restore only the folder, not its contents.

  • Undelete is only supported for the Nucleus provider.

2.21.0

• Update to carbonite 134.0

• Update omnitrace to 1.1

• OM-66254: Fix cache bypass when accessing S3 through cloudfront

• HUB-343: Fix exposing C++ symbols for unit tests

• CC-733: Add “Resolve by timestamp” feature. This allows you to specify “?timestamp={unix_time_in_seconds}” to the “stat” and “read” commands and it will select the checkpoint closest to (but not later than) the specified timestamp.

• OM-61992: Revert changes to omniClientBreakUrl. There are now 2 functions:

  1. omniClientBreakUrl should be used if you have either:

     • A URL which starts with a scheme such as “omniverse://host/path” or “http://host/path” or “file:/path”

     • A file path such as “C:/path” (windows) or “/path” (Linux)

  2. omniClientBreakUrlReference should be used if you have a URL reference such as “/path?query#fragment” which would be interpreted as a file path by omniClientBreakUrl

2.20.0

• OM-52351 Use multipart_chunk_size field from Auth response to init chunk size in LFT uploads, support for override that value using OMNI_MULTIPART_CHUNK_SIZE_MB environment variable.

• CC-763: Use new “rename2” command to support atomic checkpoints with move/rename

• HUB-336: Fixed crash when resolving if path doesn’t have any slashes

2.19.0

• Re-added python stub omni.client.set_hang_detection_time_ms to ease upgrading.

• Fixed a hang when waiting on a recurring request in Python.

• HUB-332: Added connectionId to serverInfo.

• HUB-334: Security Update zlib, libcurl, and python dependencies.

• HUB-330: Fix omniClientNormalizeUrl(“../relative/path”); on Linux

  Note that as a consequence of this change, omniClientBreakUrl will never return a “raw” file path on Windows. This matches the behavior on Linux, where it already never returned a raw file path, because it’s impossible to distinguish a raw local file path from a URL reference.

  If you know that you are passing either a URL or an absolute path, you can use omniClientBreakUrlOrPath which assumes that anything which doesn’t have a scheme is a file path.

  omniClientBreakUrl(“/local/file%20name#3.txt”) returns: path = “/local/file name” fragment = “3.txt” isRaw = false

  omniClientBreakUrlOrPath(“/local/file%20name#3.txt”) returns: path = “/local/file%20name#3.txt” isRaw = true

  But you should always prefer using “file” URLs because “file:/local/file%20name%233.txt” returns the same result when passed to either omniClientBreakUrl or omniClientBreakUrlOrPath.

2.18.0

• HUB-322: Call python callbacks from a dedicated thread so the GIL can’t block the tick thread.

• Removed warning when waiting in python. Also removed omni.client.set_hang_detection_time_ms

• HUB-316: Fix resolveSubscribe when filename is percent-encoded

• HUB-318: Update to latest python packages

2.17.0

• CC-950: Added omniClientWriteFileEx to retrieve version & hash of newly written file

• CC-969: Fix copying an empty folder on Nucleus

2.16.0

• FDN-211: Distributed tracing instrumentation for open-telemetry interoperability

• CC-920: removed an unused host dependency (“mirror”).

• CC-920: updated the repo tools versions to the latest so that they all have Mac support.

• CC-140: Add python bindings for resolve & resolve_subscribe

• CC-912: makeFileUrl should prefix ‘/’ for absolute paths on Windows.

• CC-936: Fixed crash that could happen on Windows if you stop a listSubscribe before it started.

• CC-916: Throttled folder copies to no more than 10 at a time, to prevent using tons of memory when copying large folders.

2.15.0

• CC-882: Fixed list subscribe with rename events

• CC-873: Bypass cache until client-library restarts when cache failure is seen. Provide callback to notify applications of this behavior

• CC-901: Trigger file status callbacks from the print thread to avoid Python GIL deadlocks

• CC-903: Fixed a deadlock that could occur when calling wait from inside the create object callback

2.14.0

• CC-819: Fix memory leak in _allocedRequests

• CC-345: Pass callbacks by r-value instead of const reference

• CC-812: Support “stat” on cloudfront URLs which are not configured for list

• CC-836: Add ‘download’ parameter to omniClientGetLocalFile to retrieve the filename without downloading.

• CC-771: Add “Listing” and “Stating” events to file status callback. Also trigger reading & writing events for omniClientGetLocalFile

• CC-841: Remove automatic prepend of “Copied from” & “Moved from” messages

2.13.0

• OM-57674: Update to Carbonite 128.0

• CC-735: Added python bindings for omniClientMakePrintable and updated it to also percent-decode valid utf-8 and unreserved ascii characters (previously it would percent-encode invalid characters, but not percent-decode the valid ones).

• CC-762: Automatically prepend message to checkpoint when copying & moving: “Copied from …” or “Moved from …”

2.12.0

• CC-713: Update dependencies to latest versions:

  • Python: 3.7.13, 3.8.13, 3.9.13, 3.10.5

  • OpenSSL: 1.1.1q

  • zlib: 1.2.12

  • curl: 7.83.1

  • brotli: removed (no longer needed)

• OM-54427: Add Python 3.10 builds

• CC-749: Allow multiple connections to the same host by using the “userinfo” field of the URL

2.11.1

• OM-55823: Update to connection library 113.26 to resolve a crash that could happen during disconnect while downloading a file through LFT.

2.11.0

• CC-691: increase default curl timeout to 1800, make it configurable in omniverse.toml, properly check curl error code from curl_multi handles.

• CC-693: Fall back to old copy method when copying from a mount on older servers.

• CC-534: Add ability to configure an S3 provider via omniClientSetS3Configuration

• CC-124: Add ‘bookmark’ functions: omniClientAddBookmark, omniClientRemoveBookmark, omniClientListBookmarks. The python bindings are add_bookmark, remove_bookmark, and list_bookmarks_with_callback

• CC-732: Update to connection library 113.25 to resolve a crash that could happen during disconnect while downloading a file through LFT.

2.10.0

• CC-581: Fixed setting the checkpoint message when copying on the same Nucleus server

• CC-583: Fix a case where automatic checkpoints didn’t exactly emulate atomic checkpoints. There was one case where automatic checkpoints would use the provided message for the checkpoint before and after the overwrite but atomic checkpoints use a blank message for the checkpoint prior to the overwrite.

• CC-590: Fixed resolving UDIM URIs that have been percent encoded

• CC-594: Always check that the logcallback function is valid before calling it.

• CC-566: Added documentation: http://omniverse-docs.s3-website-us-east-1.amazonaws.com/client_library/ As part of this change, we have combined a number of include files. You should now only include “OmniClient.h”

2.9.0

• CC-517: Move calling the log callback to its own thread. This moves having to acquire the GIL to call the log callback to its own thread, which can prevent deadlocks. This specifically fixes an observed deadlock with USD/usd-resolver trying to resolve several paths at once while holding the GIL. Additionaly this moves calling the log callback out of any critical paths, making the code more robust against potentially slow log callback implementations.

• CC-493: Retry without cache when cache download fails

• CC-553: Fixed two timing issues that could cause Stat/Resolve requests to hang. One affected Linux only and involved the initial callback, and the other affected both Windows and Linux and involved the subscription callback.

2.8.0

• CC-491: Switch to using copy2 function in NucleusProvider to support atomic checkpoints.

• CC-311: Map RESOURCE_BUSY error code to eOmniClientResult_ErrorLocked

• CC-501: Add omniClientBreakUrlAbsolute & omni.client.break_url_absolute

• OM-46177: Fix “checkpoint_version” could be stopped before it’s ever sent to the server

• OM-49835: Speculative fix for a crash on shutdown

• CC-504: Add message parameter to omniClientCopy, omniClientMove, and omniClientWriteFile that allows users to set the message field of the atomic checkpoints created by those functions.

• OM-47306: Print a warning (and a traceback) when python calls wait() and it takes longer than 20ms

2.7.0

• OM-47514: Calling stat on a checkpoint will no longer have the “writable file” flag set

• CC-476: Add user/group mangement python bindings

• OM-35591: supporting atomic checkpoints

• CC-424: Make NucluesProviderFactory::initializeConnectionLibrary() thread safe.

• CC-482: Change shutdown warning to only print after Core::Shutdown is complete.

• CC-436: Fix URL parsing methods to not percent-decode raw file paths

• OM-49023: Properly handle large reads and writes in windows and linux file providers.

2.6.0

• CC-333: Removed USD plugin (moved to OmniUsdResolver)

• CC-397: Fixed makeRelative when there are percent-encoded urls

• CC-406: Add Python 3.8 and 3.9 builds

• CC-407: Fix listcache going out of date if the server sent a meta event after a deleted event

• CC-437: Add python bindings for omniClientLiveSetQueuedCallback

• OM-47554: Downgrade some “Error” messages during auth to “Verbose” We attempt to connect using 3 different methods, and it’s normal for 2 of them to fail. This avoids confusing users with error messages for a connection that ultimately succeeds.

• OM-48252: Lower required “list2” version, to allow connecting to servers running Nucleus 112.0.

• OM-28724: Retry up to 3 times when there’s an LFT error

Notes for updating:

1. If you are working with USD, you should no longer pull this omni_client_library directly anymore.

Instead, pull the package “usd-resolver”:

  <dependency name="omni_usd_resolver" linkPath="../_build/target-deps/omni_usd_resolver">
    <package name="omni_usd_resolver.nv-20_08.py37.${platform}" version="1.0.0" />
  </dependency>

This includes a file named “deps/redist.packman.xml” that you can include in your own packman.xml to grab the exact version of USD and the client library that the resolver was built against. Note: This requires packman 6.4 or later!

  <project toolsVersion="5.0">
    <import path="../_build/target-deps/omni_usd_resolver/deps/redist.packman.xml">
    </import>
    <dependency name="usd_debug" linkPath="../_build/target-deps/usd/debug">
    </dependency>
    <dependency name="usd_release" linkPath="../_build/target-deps/usd/release">
    </dependency>
    <dependency name="omni_client_library" linkPath="../_build/target-deps/omni_client_library">
    </dependency>
  </project>

Ensure your build scripts also copy from (or refrence) the binaries in omni_usd_resolver.

NOTE: omniClientInitialize no longer registers the Omniverse USD Resolver plugin! You must either copy the omni_usd_resolver plugin in the default USD plugin location, or register the plugin location at application startup using PXR_NS::PlugRegistry::GetInstance().RegisterPlugins.

See the README.md in omni_usd_resolver for more information about how to set this up.

2. “OmniUsdLive.h” was renamed to “OmniClientLive.h” C: All the function prefixes were changed from “omniUsdLive” to “omniClientLive” Py: All the function prefixes were changed from “omni.client.usd_live” to “omni.client.live”

3. There are no longer specific python flavors of client library. Instead, it ships with bindings for Python 3.7, 3.8, and 3.9 all in one package. In packman.xml, change “omni_client_library.py37.${platform}” to “omni_client_library.${platform}” This is only relevant if you are NOT getting this omni_client_library from the omni_usd_resolver.

2.5.0

• CC-357: Fixed a deadlock that could occur when a python file status callback is being unregistered on one thread while another thread is simultaneously trying to call that file status callback

• CC-367: Allow stat & list using cloudfront.net URLs

• CC-384: Remove support for Nucleus 109 & 110

• OM-45887: Enable extra connection logging

• CC-366: Update OpenSSL to avoid a security vulnerability

• CC-380: Fix SdfLayer::OpenAsAnonymous with “omniverse” URLs

2.4.0

• OM-43249: Fixed crash that could occur when authentication result arrives after authentication was cancelled

• OM-23151: Allow multiple authentication requests to be in flight at the same time This changes:

  • C: omniClientSetAuthenticationMessageBoxCallback

  • C: omniClientAuthenticationCancel

  • py: authentication_cancel

  • py: set_authentication_message_box_callback

• CC-339: omniClientMakeRelative returns relative paths with ‘/’ even on Windows

• CC-343: Files inside a mount should not have fOmniClientItem_WriteableFile

• CC-186: Add support for moving/renaming files & folders

  • C: omniClientMove

  • Py: omni.client.move

  • Py: omni.client.move_async

  • Py: omni.client.move_with_callback

• CC-219: Add user/group management for nucleus provider (other providers do not support user management)

2.3.0

• CC-166: Trigger omniClientStatSubscribe callback when .live files change

• CC-236: Add support for checkpoint change notification

2.2.0

• CC-245: Fixed a deadlock that could occur rarely when connection failed.

• CC-221: Upgrade OmniTrace to 0.5.8831531e for correct libstdc++ license

• CC-32: Fixed a crash that could happen on disconnect

• OM-43009: Removed “stop” from list of required capabilities, to discover newer servers which don’t advertise that capability

• CC-271: Removed support for Nucleus 107 (released in Nov 2019)

2.1.0

• CC-228: Subscription based authentication using ‘nonce’

• CC-231: Discover API server using minimum required capabilities rather than all capabilites

• CC-239: Fixed an issue where log messages issued when omniclient.dll is unloading could access invalid memory.

• OM-41660: updating the connection library which was build with a proper toolchain (licensing fixes)

• CC-225: Fixed a rare crash that could happen when disconnecting from a Nucleus server.

• CC-123: Add support for deleting a checkpoint

2.0.0

Big Live Mode Changes!

You can no longer enable live mode on any arbitrary layer. Instead, layers with the “.live” extension are considered live layers. Changes made to a live layer are replicated to other users when you call omniUsdLiveProcess (py: usd_live_process). Calling Save on a live layer does nothing.

This is not a backwards-compatible change. All clients must be using either client library version 1 or version 2 in order to do live-syncing with each other.

New (v2) clients can read & write USD files on old servers, but require a Nucleus server 111 or later in order to do live updates.

You can detect if a server supports the new live update method by calling “omniClientGetServerInfo” (py: omni.client.get_server_info) and checking the “omniObjectsEnabled” (py: omniojects_enabled) field.

Live files show up in “list” with the flags “fOmniClientItem_IsOmniObject” and “fOmniClientItem_CanLiveUpdate” (py: IS_OMNI_OBJECT | CAN_LIVE_UPDATE).

Copying a .live file from one Nucleus server to the same server or a different Nucleus server is supported, but copying it to a local drive or other provider types (http, for example) is not.

Removed the following functions:

• omniUsdLiveSetDefaultEnabled & omniUsdLiveGetDefaultEnabled

• omniUsdLiveSetModeForUrl & omniUsdLiveGetModeForUrl

• omniUsdLiveSetResolveMethod & omniUsdLiveGetResolveMethod

The equivalent python bindings have also been removed:

• usd_live_set_default_enabled & usd_live_get_default_enabled

• usd_live_set_mode_for_url & usd_live_get_mode_for_url

Other Changes:

• CC-223: Fixed some cases where the list cache would be invalidated at the wrong time

• CC-156: Improve Tokens Caching (cache moved to “auth.toml” and its structure changed to use auth. transport info)

• CC-229: Fixed a case where the client library would not connect to Nucleus securely