Tutorial - OmniGraph Build Environment Conversion

These steps will set up the extension to enable building the .ogn file generation and access the generated tests, include file, and Python file. This only has to be done once per extension, after which any newly converted or created nodes will be automatically picked up by the build.

Once the build has been set up any existing nodes can be converted to the .ogn format using the steps described in Tutorial - OmniGraph .ogn Conversion.

Note

If you are building outside Kit you will have to add access to the OGN processing tools in your main premake5.lua file. See Tutorial - Building OmniGraph Outside Kit for more details.

Step 1 : File Structure

These are a set of guidelines for file locations and contents that make things consistent, which in turn makes the lives of developers easier when looking at different sections of code. See Recommended Directory Structure for more information on why the directory is structured this way.

Here is the file structure of the extension containing the jiggle node before conversion:

omni.anim.jiggle/
├── bindings/
|   └── BindingsPython.cpp
├── config/
|   └── extension.toml
├── plugins/
|   ├── Deformer.cpp
|   ├── IJiggleDeformer.h
|   └── JiggleDeformer.cpp
├── premake5.lua
└── python/
    ├── __init__.py
    └── scripts/
        ├── commands.py
        ├── extension.py
        ├── jiggleDeformer.py
        ├── kit_utils.py
        ├── menu.py
        └── test.py

Python Scripts

First thing to note is that we’d like to keep the utility scripts in the python subdirectory separate from any tests. The directories can be rearranged with a simple git command:

git mv python/scripts/tests.py python/tests

That leaves the python directory with this structure:

python/
├── __init__.py
└── scripts/
    ├── commands.py
    ├── extension.py
    ├── jiggleDeformer.py
    ├── kit_utils.py
    └── menu.py
└── tests/
        └── test.py

Note

The reason that scripts/ is a separate subdirectory is to allow the Python code to be linked as a whole in the build rather than copied. That way you can drop any file into the scripts/ directory in your source tree, edit it in place, and have the extension system hot-reload it so that the new script is immediately available while the app is running.

Nodes

There is no restriction on how many nodes an extension can support though in this example there is only one. Each node will have two files; one with the suffix .ogn, containing the node description, and one with the suffix .cpp, containing the node class.

It is helpful to keep the node naming and location consistent. The convention established is to use Ogn as a prefix to the class name, and to put the node files in a nodes/ directory, parallel to plugins/. In the example here the means creating a new directory and renaming the node as follows:

mkdir nodes
git mv plugins/JiggleDeformer.cpp nodes/OgnJiggleDeformer.cpp
touch nodes/OgnJiggleDeformer.ogn

The last command creates a placeholder for the file to be populated later.

At this point all of the files are in their final location and the directory structure changes look like this:

omni.anim.jiggle/                           omni.anim.jiggle/
├── bindings/                               ├── bindings/
|   └── BindingsPython.cpp                  |   └── BindingsPython.cpp
├── config/                                 ├── config/
|   └── extension.toml                      |   └── extension.toml
├── nodes/                                  ├── plugins/
|   ├── OgnJiggleDeformer.cpp               |   ├── JiggleDeformer.cpp
|   └── OgnJiggleDeformer.ogn               |   |
├── plugins/                                |   |
|   ├── Deformer.cpp                        |   ├── Deformer.cpp
|   └── IJiggleDeformer.h                   |   └── IJiggleDeformer.h
├── premake5.lua                            ├── premake5.lua
└── python/                                 └── python/
    ├── __init__.py                             ├── __init__.py
    ├── scripts/                                └─── scripts/
    |   ├── commands.py                             ├── commands.py
    |   ├── extension.py                            ├── extensions.py
    |   ├── jiggleDeformer.py                       ├── jiggleDeformer.py
    |   ├── kit_utils.py                            ├── kit_utils.py
    |   ├── menu.py                                 ├── menu.py
    └── tests/                                      |
        └── test.py                                 └── test.py

Step 2 : Fix Python Imports

Since the python files have move the python import directives have to be updated to point to the new locations. Most of them are okay as they are relative, only those whose relative position will change need to be adjusted such as the import of the bindings from extension.py. In addition, the generated ogn/ subdirectory must also be imported if you have any nodes implemented in Python.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import omni.ext
from .bindings._anim_jiggle import *

from .menu import DeformerMenu
from .commands import *
from .test import *


class Extension(omni.ext.IExt):
    def on_startup(self):
        self._ext = acquire_interface()
        self._menu = DeformerMenu(self._ext)

    def on_shutdown(self):
        release_interface(self._ext)
        self._menu.on_shutdown()
        self._menu = None

The bindings will now be at the same relative level so the first highlighted line changes to:

from .bindings._anim_jiggle import *

and the tests will be automatically detected via a mechanism to be added later, Step 5 : Add Python Initialization, so the second highlighted line will be deleted.

Also, due to the changing locations, the import of extension.py in the file __init__.py changes from this:

from .scripts.extension import *

to this

from .extension import *

Step 3 : Add Build Support

The OmniGraph nodes, and the .ogn files in particular, require some build support in order to be processed correctly.

Note

This process assumes you are using the *extension 2.0* plugin interface

Here are the original contents of the Jiggle extension’s premake5.lua file, with the areas that will be modified highlighted:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
local ext = get_current_extension_info()

-- change name of plugins here
local namespace = "omni/anim/jiggle"

-- lines below can be copied for other plugins
local python_source_path = "python"
local plugin_source_path = "plugins"

ext.group = "animation"
project_ext(ext)


project_ext_plugin(ext, "omni.anim.jiggle.plugin")

    add_files("impl", "plugins")
    add_files("iface", "%{root}/include/"..namespace)

    exceptionhandling "On"
    rtti "On"
    removeflags { "NoPCH" } -- to speed up USD includes compiling
    filter {}
    includedirs {
        "%{target_deps}/nv_usd/%{config}/include",
        "%{target_deps}/rtx_plugins/include",
        "%{target_deps}/cuda/include"
    }
    filter { "system:linux" }
        removeflags { "FatalCompileWarnings", "UndefinedIdentifiers" }
        includedirs { "%{target_deps}/python/include/python3.7m" }
    filter { "system:windows" }
        libdirs { "%{target_deps}/tbb/lib/intel64/vc14" }
    filter {}
    libdirs { "%{target_deps}/nv_usd/%{config}/lib" }
    links {"ar","vt", "gf", "pcp", "sdf", "arch", "usd", "tf", "usdUtils", "usdGeom", "usdSkel", "omni.usd"}
    filter {}


project_ext_bindings {
    ext = ext,
    project_name = "omni.anim.jiggle.python",
    module = "_anim_jiggle",
    src = "bindings",
    target_subdir = namespace.."/bindings"
}
    vpaths {
        ['bindings'] = "bindings/*.*",
        ['python'] = python_source_path.."/*.py",
        ['python/scripts'] = python_source_path.."/scripts/*.py"
    }
    files {
        python_source_path.."/*.py",
        python_source_path.."/scripts/*.py"
    }

    includedirs { plugin_source_path }

    repo_build.prebuild_link {
        { "python/scripts", ext.target_dir.."/"..namespace.."/scripts" },
    }


    repo_build.prebuild_copy {
        { "python/*.py", ext.target_dir.."/"..namespace },
    }

Set up Helper Variables With Paths

The first change is to add a few helper variables containing directories of interest (fixing the generic copy/pasted comments while we are in there). Similar to the get_current_extension_info() helper function there is get_ogn_project_information() that defines a table of useful information for OGN projects, assuming a common layout. See below for details on what it returns.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
local ext = get_current_extension_info()

-- Helper variable containing standard configuration information for projects containing OGN files
local ogn = get_ogn_project_information(ext, "omni/anim/jiggle")

-- Grouping also tells the name of the premake file that calls this one - $TOP/premake5-{ext.group}.lua
ext.group = "animation"

-- Set up common project variables
project_ext( ext )

The usage of the ogn table will be illuminated below, as will the function definition that was loaded from omni.graph.tools.

This is extracted from the documentation of the get_ogn_project_information() function:

-- These are the generated table contents:
--    Source Paths
--      bindings_path : Subdirectory containing the C++ implementing Python bindings
--      docs_path     : Subdirectory containing the documentation to publish
--      nodes_path    : Subdirectory containing the C++ OmniGraph nodes
--      plugin_path   : Subdirectory containing the C++ implementing the plugin
--      python_path   : Subdirectory containing the Python code
--    Target Paths
--      python_target_path       : Path to the Python import directory of the extension
--      python_tests_target_path : Path to the Python tests import directory of the extension
--      bindings_module          : Name of the Python bindings module for the extension
--      bindings_target_path     : Pyton bindings module import path, relative to root of the extension's Python tree
--    Project Information
--      import_path    : Subdirectory for both Python and C++ import/include
--      namespace      : Prefix for project names (import_path with "/" replace by ".")
--      plugin_project : Name of the project that builds the C++ code
--      ogn_project    : Name of the project that processes the .ogn files
--      python_project : Name of the project that processes the Python code

Add the .ogn files to the project

Although this project doesn’t use the .ogn files directly it’s useful to have them available in the project file. Comments are also added while in here to make it clear what each section is doing.

1
2
3
4
5
6
-- ----------------------------------------------------------------------
-- Define the project containing the C++ code
project_ext_plugin(ext, ogn.plugin_project)

    add_files("impl", ogn.plugin_path)
    add_files("ogn", ogn.nodes_path)

The reference to the iface section was removed as it didn’t contain any files and under extension 2.0 that is an error.

Add the generated code to the include path

The generated files must be accessible to the project so a helper function add_ogn_dependencies() was added to make this common set of dependencies easy to implement. It handles includedirs, libdirs, links, and dependson relationships needed by the OGN component of the project, as well as common project compiler configuration. Thanks to those definitions the rest of the project is pared down to this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
-- Add the standard dependencies all OGN projects have
add_ogn_dependencies(ogn)

includedirs {
    "%{target_deps}/rtx_plugins/include",
    "%{target_deps}/cuda/include"
}

filter { "system:linux" }
    removeflags { "FatalCompileWarnings", "UndefinedIdentifiers" }
    includedirs { "%{target_deps}/python/include/python3.7m" }
filter { "system:windows" }
    libdirs { "%{target_deps}/tbb/lib/intel64/vc14" }
filter {}

links {"ar", "vt", "gf", "pcp", "sdf", "arch", "usd", "tf", "usdUtils", "usdGeom", "usdSkel", "omni.usd"}

Insert .ogn project

The .ogn processing must precede the compiling of files including the generated headers so it must always happen first. Unfortunately the current build system doesn’t allow for individual file dependencies like that so instead a secondary project must be set up that will process your .ogn files. The easiest place to insert it is just before the Python project definition:

1
2
3
-- ----------------------------------------------------------------------
-- Breaking this out as a separate project ensures the .ogn files are processed before their results are needed
project_ext_ogn( ext, ogn )

Notice the ogn table defined earlier being passed in to the project definition function.

Update extension definition

In some cases this may not be necessary but our example extension uses the old extension definition for its Python files as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
project_ext_bindings {
    ext = ext,
    project_name = namespace..".python",
    module = "_anim_jiggle",
    src = "bindings",
    target_subdir = import_path.."/bindings"
}
    vpaths {
        ['bindings'] = "bindings/*.*",
        ['python'] = python_source_path.."/*.py",
        ['python/scripts'] = python_source_path.."/scripts/*.py"
    }
    files {
        python_source_path.."/*.py",
        python_source_path.."/scripts/*.py"
    }

This combination of vpaths and files definitions is replaced under extension 2.0 with the following, also adding the new python/tests directory to the definition:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
project_ext_bindings {
    ext = ext,
    project_name = python_project,
    module = "_anim_jiggle",
    src = "bindings",
    target_subdir = import_path.."/bindings"
}

    -- All Python script and documentation files are part of this project
    add_files("bindings", "bindings/*.*")
    add_files("python", "python/*.py")
    add_files("python/scripts", "python/scripts/**.py")
    add_files("python/tests", "python/tests/**.py")

The parameters to the project can be replaced with the ones defined in the ogn table:

1
2
3
4
5
6
7
8
9
-- ----------------------------------------------------------------------
-- Python support and scripts for the Jiggle deformer
project_ext_bindings {
    ext = ext,
    project_name = ogn.python_project,
    module = ogn.bindings_module,
    src = ogn.bindings_path,
    target_subdir = ogn.bindings_target_path
}

Set up OGN Dependencies for Python Nodes

Python nodes will function best if they have hot reloading capability (i.e. a change you make to the source file will be immediately reflected in the running application). The function that sets up project dependencies for OGN, add_ogn_dependencies has an extra parameter that is a table of directories in which Python OGN nodes live.

This is the directory where your OGN node files are named OgnMyNode.py. While it’s helpful to have the .ogn file in the same directory it’s not necessary for this particular purpose.

By adding the relative path to all directories containing these nodes the extension configuration will automatically pick them up and register the nodes with OmniGraph, as well as automatically updating them at runtime if the source files change. Any future nodes added to the named directories will work with no further change to the build file.

The directory is linked in the build, not copied, so it picks up all subdirectories as well. The only time you need to modify this table is if a new directory containing Python nodes is created that is a sibling to the currently named directories. As the registration looks for the node file name pattern of Ogn.*.py you can put other files in the directory without causing any registration problems. You could even include your top level python/ directory to ensure all Python nodes are picked up.

1
2
-- Add the standard dependencies all OGN projects have, and link directories with Python nodes
add_ogn_dependencies(ogn, {"python/nodes"})

Step 4 : Add Extension Support

The build system handles the compile-time support for the nodes, the extension support will add the link-time support. For extension 2.0 this exists in the form of the file config/extension.toml. It must be taught about the python test modules and the extension dependencies.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[package]
title = "Omniverse Jiggle Deformer"

# Main module of the python interface
[[python.module]]
name = "omni.anim.jiggle"

# Additional python module used to make .ogn test files auto-discoverable
[[python.module]]
name = "omni.anim.jiggle.ogn.tests"

# Watch the .ogn files for hot reloading (only for Python files)
[fswatcher.patterns]
include = ["*.ogn", "*.py"]
exclude = ["Ogn*Database.py"]

# Other extensions that must be loaded before this one
[dependencies]
"omni.graph" = {}
"omni.graph.tools" = {}
"omni.kit.usd_undo" = {}
# The generated tests will make use of these modules
"omni.usd" = {}
"omni.kit.async_engine" = {}

[[native.plugin]]
path = "bin/${platform}/${config}/*.plugin"
recursive = false

Step 5 : Add Python Initialization

Even nodes written entirely in C++ will have Pythonic interfaces, and possibly Python test scripts. It’s best if you initialize the OGN code on startup in order to take advantage of it rather than trying to do it manually later.

So long as you have an ogn/ subdirectory in your Python import path this will all be taken care of automatically for your; both registration and deregistration of your nodes when the extension loads and unloads respectively.

Step 6 : Set up automatic test detection (optional)

While you could set up test inclusion manually, you would have to add every new test file you create as well. A mechanism was created to automatically scan a directory for test files and add them to the list testrunner uses for regression tests. This operates in a similar manner to how the build looks for test files, but at runtime rather than build time.

To enable this test detection, create python/tests/__init__.py with the content below. (Other content is fine if you need it for other tests, this code can appear anywhere in that file.)

"""
Presence of this file allows the tests directory to be imported as a module so that all of its contents
can be scanned to automatically add tests that are placed into this directory.
"""
scan_for_test_modules = True

Now that the build can handle the .ogn files, existing nodes can be converted to the .ogn format using the steps described in Tutorial - OmniGraph .ogn Conversion.

Final Result - The Converted Build Script

As reference, here is the final version of the file omni.anim.jiggle/premake5.lua with all new lines highlighted:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
local ext = get_current_extension_info()

-- Helper variable containing standard configuration information for projects containing OGN files
local ogn = get_ogn_project_information(ext, "omni/anim/jiggle")

-- Grouping also tells the name of the premake file that calls this one - $TOP/premake5-{ext.group}.lua
ext.group = "animation"

-- Set up common project variables
project_ext( ext )

-- ----------------------------------------------------------------------
-- Define the project containing the C++ code
project_ext_plugin(ext, ogn.plugin_project)

    add_files("impl", ogn.plugin_path)
    add_files("ogn", ogn.nodes_path)

    -- Add the standard dependencies all OGN projects have
    add_ogn_dependencies(ogn)

    includedirs {
        "%{target_deps}/rtx_plugins/include",
        "%{target_deps}/cuda/include"
    }

    filter { "system:linux" }
        removeflags { "FatalCompileWarnings", "UndefinedIdentifiers" }
        includedirs { "%{target_deps}/python/include/python3.7m" }
    filter { "system:windows" }
        libdirs { "%{target_deps}/tbb/lib/intel64/vc14" }
    filter {}

    links {"ar", "vt", "gf", "pcp", "sdf", "arch", "usd", "tf", "usdUtils", "usdGeom", "usdSkel", "omni.usd"}


-- ----------------------------------------------------------------------
-- Breaking this out as a separate project ensures the .ogn files are processed before their results are needed
project_ext_ogn( ext, ogn )


-- ----------------------------------------------------------------------
-- Python support and scripts for the Jiggle deformer
project_ext_bindings {
    ext = ext,
    project_name = ogn.python_project,
    module = ogn.bindings_module,
    src = ogn.bindings_path,
    target_subdir = ogn.bindings_target_path
}

    -- All Python script and documentation files are part of this project
    add_files("bindings", "bindings/*.*")
    add_files("python", "python/*.py")
    add_files("python/tests", "python/tests/**.py")

    -- Add the standard dependencies all OGN projects have, and link directories with Python nodes
    add_ogn_dependencies(ogn, {"python/nodes"})

    -- Copy the init script directly into the build tree to avoid reload conflicts.
    repo_build.prebuild_copy {
        { "python/__init__.py", ogn.python_target_path },
    }
    -- Linking directories allows them to hot reload when files are modified in the source tree
    repo_build.prebuild_link {
        { "python/scripts", ogn.python_target_path.."/scripts" },
        { "python/tests", ogn.python_tests_target_path },
    }