AutoNode - Autogenerating Nodes From Code

This is a description of the automated node generator for python code in OmniGraph.

It turns this:

import omni.graph.core as og

@og.AutoFunc(pure=True)
def dot(vec1: og.Float3, vec2: og.Float3) -> float:
    return vec1[0] * vec2[0] + vec1[1] * vec2[1] + vec1[2] * vec2[2]

Into this:

What it Does

AutoNode allows developers to create OmniGraph nodes from any of the following:

• Free functions (see AutoFunc)

• Python classes and class instances - AutoNode also scans classes and generates getters and setters (see AutoClass and Passing Python Types in AutoNode ).

• CPython and Pybind types - see Annotations: Overriding CPython Types.

• OmniGraph types (see Supported Types )

• Container types (see Bundles)

• Events, Enums and other special types.

This process will generate both the OGN description of the node and its attributes, and the implementation of the node compute function in Python.

How it Works

AutoNode generates node signatures by extracting type annotations stored in functions and variables. In order for nodes to be generated, type annotations must be available to the decorator at initialization time. For most developers, this means using type annotations for python 3, like in the example above.

Under the hood, annotation extraction is done with the python __annotations__ ( PEP 3107 ) dictionary in every class.

Note

While the API will remain relatively stable, there is no current guarantee that the backend will not be changed. Some implementation details are captured in Implementation Details, purely for reference.

API Details

Decorators

omni.graph.core currently exposes these autogeneration functions: AutoFunc(), AutoClass(). Both these functions can be used as decorators or as free function calls:

import omni.graph.core as og

#this is convenient
@og.AutoFunc()
def say_hi() -> str:
    return "hi, everybody!"

def say_hello() -> str:
    return "Hello, World!"

# but this does virtually the same thing
og.AutoFunc()(say_hello)

AutoFunc

Here’s an example of wrapping a free function:

import omni.graph.core as og

@og.AutoFunc()
def one_up(input : int = 0) -> str:
"""this documentation will appear in the node's help"""
    return f"one up! {input + 1}"

Note: The type annotations really matter. Without them, AutoNode can’t define node inputs and outputs.

You can create multiple returns by making your return type a typing.Tuple:

import omni.graph.core as og
from typing import Tuple

@og.AutoFunc()
def break_vector(input : og.Float3) -> Tuple[og.Float, og.Float, og.Float]
    return (input[0], input[1], input[2])

AutoClass

This exposes entire classes to OGN. Here are the rules of evaluation:

• Private properties and methods (starting with _) will not be exposed

• Public properties exposed will attempt to find a type annotation using __annotations__ and will default to the inferred type. None types will be ignored.

• Public methods will be associated with the class type, and will not attempt to find a type for self.

• Bound methods in classes (methods with an immutable __self__ reference) will be stored for representation, but the actual method to be called will come from the specific instance that’s being applied for this. For instance - if class A was decorated with AutoClass, and it contained a method A.Method(self, arg: int)->str, with __self__ stored in the method (“bound method”), then when a class B with an overriding method gets called on this node, the node will search for inside B.Method and call it instead.

Here is a simple case of wrapping a class:

import omni.graph.core as og

@og.AutoClass(module_name="AutoNodeDemo")
class TestClass:

    #this will not generate getters and setters
    _private_var = 42

    #this will generate getters and setters, and will make them of type ``float``
    public_var: float = 3.141

    # initializers are private methods and will not be exposed by the decorator
    def __init__(self):
        pass

    def public_method(self, exponent: float) -> float:
        """this method will be exposed into OmniGraph, along with its'
        documentation here. The function signature is necessary on every
        argument except for ``self``, which is implied."""
        return self.public_var ** exponent

This results in the following menu and implementation:

Note that getters and setters take in a target variable. This allows users to pass in a reference to an existing class. See Passing Python Types in AutoNode for more details.

Decorator Parameters

pure

Boolean. Setting @og.AutoFunc(pure=True) will generate a method that can be used in a push or execute context, without an execution input or output. AutoFunc only.

For example, these two functions:

import omni.graph.core as og

@og.AutoFunc(pure=True)
def dot(vec1: og.Float3, vec2: og.Float3) -> float:
    return vec1[0] * vec2[0] + vec1[1] * vec2[1] + vec1[2] * vec2[2]


@og.AutoFunc()
def exec_dot(vec1: og.Float3, vec2: og.Float3) -> float:
    return vec1[0] * vec2[0] + vec1[1] * vec2[1] + vec1[2] * vec2[2]

Will generate these nodes:

module_name

Specifies where the node will be registered. This affects UI menus and node spawning by name. For example, setting

import omni.graph.core as og
import MyModule

@og.AutoFunc(module_name="test_module")
def add_one(input: int) -> int:
    return input + 1

og.AutoClass(module_name="test_module")(MyModule.TestClass)

Will result in a UI displaying access to this node from test_module

ui_name

Specifies the onscreen name of the function. AutoFunc only.

annotation

See Annotations: Overriding CPython Types.

tags

Array of text tags to be added to the node.

metadata

Other metadata to pass through AutoNode, such as icon information.

Supported Types

Currently, this is the list of types supported by AutoNode:

import omni.graph.core as og

# Generic Vectors
[og.Vector3d, og.Vector3h, og.Vector3f]

# Scalar and Vector types
[og.Float, og.Float2, og.Float3, og.Float4]
[og.Half, og.Half2, og.Half3, og.Half4]
[og.Double, og.Double2, og.Double3, og.Double4]
[og.Int, og.Int2, og.Int3, og.Int4]

# Matrix Types
[og.Matrix2d, og.Matrix3d, og.Matrix4d]

# Vector Types with roles
[og.Normal3d, og.Normal3f, og.Normal3h]
[og.Point3d, og.Point3f, og.Point3h]
[og.Quatd, og.Quatf, og.Quath]
[og.TexCoord2d, og.TexCoord2f, og.TexCoord2h]
[og.TexCoord3d, og.TexCoord3f, og.TexCoord3h]
[og.Color3d, og.Color3f, og.Color3h, og.Color4d, og.Color4f, og.Color4h]

# Specialty Types
[og.Timecode, og.Uint, og.Uchar, og.Token]

# Underspecified, but accepted types
# str, int, float

Note

Not all python typing types are supported.

Bundle Types : The Omniverse struct

Since most of Omniverse’s computations require concrete numerical types - matrices, vectors, numbers with known precision - it makes sense to have a data structure to pass them in together - and to make that data structure GPU-friendly. Bundle serves that purpose - it’s a lightweight data structure which contains information about the types it holds.

Bundle can be passed around between nodes in OmniGraph without special handling. Here is one example:

import omni.graph.core as og

@og.AutoFunc(pure=True)
def pack(v: og.Float3, c: og.Color3f) -> og.Bundle:
    bundle = og.Bundle("return", False)
    bundle.create_attribute("vec", og.Float3).value = v
    bundle.create_attribute("col", og.Color3f).value = c
    return bundle


@og.AutoFunc(pure=True)
def unpack_vector(bundle: og.Bundle) -> og.Float3:
    vec = bundle.attribute_by_name("vec")
    if vec:
        return vec.value
    return [0,0,0]

Will yield this:

Bundles aren’t just useful as ways to easily pass data structures into the graph - they’re also useful within python. Passing bundles between python methods is relatively cheap, and allows all users of the data to know the exact type and size it was used in. Furthermore, usage of bundles allows OmniGraph to move data to the GPU at will.

Passing Python Types in AutoNode

While OmniGraph can only understand and manipulated the types listed above, any python type can be passed through OmniGraph by AutoNode. This is done by using a special type called objectId, which is used by AutoNode to keep a reference (usually a hash, but that can change, see Implementation Details) to the object being passed through OmniGraph.

Any type in a method I/O that isn’t recognized by AutoNode as supported by OmniGraph is automatically handled by AutoNode like this:

1. When a method signature has a type not recognized by AutoNode, AutoNode generates an objectId signature for that type in the node’s ogn descriptor.

2. When a method outputs a object whose type is represented by an objectId in the output, AutoNode stores the object in a Temporary Object Store, a special runtime collection of python objects resident in OmniGraph memory.

3. When an input objectId is received by a method, the method requests the object from the Temporary Object Store. The object store is “read once” - after a single read, the object is deleted from the store. Every subsequent call will result in an error.

Special Types

AutoNode adds special type handlers (see Adding Type Handlers) to deal with how certain types are used in practice.

Enums

Type handlers which specialize enums are built for how enums are typically used: Switching on an Enum, like an event type. The switch class has one exec input and one enum input, and exec outputs to match the number of enum members:

from enum import Enum

@AutoClass()
class DogType(Enum):
    SHIBA = 1
    HUSKY = 2
    SALUKI = 4

Results in:

Events

TODO

Annotations: Overriding CPython Types

Sometimes, importing modules created in CPython is essential. Those modules may not have a __annotations__ property, so we can create it for them. Suppose the codeblocks above were defined in CPython and didn’t contain annotations. Here we create a shim for them:

import omni.graph.core as og
import MyModule

# Note that we don't need a decorator call, since the function is being passed as an argument

og.AutoFunc(annotation={
    "input": int,
    "return": str
    })(MyModule.one_up)

Similarly, for classes:

import omni.graph.core as og
import MyModule

# Note that members only get a type annotation, but functions get the entire annotation.

og.AutoClass( annotation={
        "public_var" : float,
        "public_method": {
            "self": MyModule.TestClass,
            "exponent": float,
            "return": float}
    })(MyModule.TestClass)

Customizing AutoNode

Adding Type Handlers

Types going through AutoClass are highly customizable. By default, when a class is scanned, it is run through a series of custom handlers, each of type AutoNodeDefinitionGenerator. This class implements this interface:

import omni.graph.core as og

# ================================================================================
class MyDefinitionGenerator(og.AutoNodeDefinitionGenerator):
    """Defines a property handler"""

    # Fill this with the type you wish to scan
    _name = "MyDefinition"

    # --------------------------------------------------------------------------------
    @classmethod
    def generate_from_definitions(cls, new_type: type) -> Tuple[Iterable[og.AutoNodeDefinitionWrapper], Iterable[str]]:
        '''This method scans the type new_type and outputs an AutoNodeDefinitionWrapper from it, representing the type,
        as well as a list of members it wishes to hide from the rest of the node extraction process.

        Args:
            new_type: the type to analyze by attribute

        Returns: a tuple of:
            Iterable[AutoNodeDefinitionWrapper] - an iterable of AutoNodeDefinitionWrapper - every node wrapper that is
                generated from this type.
            Iterable[str] - an iterable of all members covered by this handler that other handlers should ignore.
        '''
        pass

The emitted AutoNodeDefinitionGenerator class has the following signature:

import omni.graph.core as og

class MyNodeDefinitionWrapper(og.AutoNodeDefinitionWrapper):
    """Container for a single node representation consumed by the Ogn code generator.
    Class is abstract and meant to be overridden. A sufficient implementation overrides these methods:
    * get_ogn(self) -> Dict
    * get_node_impl(self)
    * get_unique_name(self) -> str
    * get_module_name(self) -> str
    """
    def __init__(self):
        super().__init__()

    # --------------------------------------------------------------------------------
    def get_ogn(self) -> Dict:
        """Get the Ogn dictionary representation of the node interface.
        Overrides the AutoNodeDefinitionWrapper function of the same name.
        """
        return {}

    # --------------------------------------------------------------------------------
    def get_node_impl(self):
        """Returns the Ogn class implementing the node behavior. See the OmniGraph documentation on how to implement.
        A sufficient implementation contains a staticmethod with the function: compute(db)
        Overrides the AutoNodeDefinitionWrapper function of the same name.
        """
        return None

    # --------------------------------------------------------------------------------
    def get_unique_name(self) -> str:
        """Get nodes unique name, to be saved as an accessor in the node database.
        Overrides the AutoNodeDefinitionWrapper function of the same name.

        Returns:
            the nongled unique name
        """
        return ""

    # --------------------------------------------------------------------------------
    def get_module_name(self) -> str:
        """Get the module this AutoNode method was defined in.
        Overrides the AutoNodeDefinitionWrapper function of the same name.

        Returns:
            the module name
        """
        return ""

For more information on node implementations requires for get_node_impl, read Tutorial 11 - Complex Data Node in Python.

Here is an example of creating an extension for enum types:

from typing import Dict, OrderedDict, Tuple, Iterable
import omni.graph.core as og

# ================================================================================
class OgnEnumExecutionWrapper:

    # We use the __init_subclass__ mechanism to execute operations on a type
    def __init_subclass__(cls, target_class: type) -> None:

        # __members__ are the property required for identifying an enum
        cls.member_names = [name for name in target_class.__members__]
        cls.value_to_name = {
            getattr(target_class, name): name for name in target_class.__members__}

    # --------------------------------------------------------------------------------
    @classmethod
    def compute(*args):
        cls = args[0]
        db = args[1]

        # don't execute if there's no need
        if not db.inputs.exec:
            return True

        input = db.inputs.enum
        input_value = og.TypeRegistry.instance().refs.pop(input)
        name = cls.value_to_name.get(input_value.value, None)
        if name is not None:
            out = getattr(db.outputs.attributes, name)
            db.outputs.context_helper.set_attr_value(True, out)
            return True

        return False


# ================================================================================
class OgnEnumWrapper(og.AutoNodeDefinitionWrapper):
    """Wrapper around Enums"""

    def __init__(
            self,
            target_class,
            unique_name: str,
            module_name: str,
            *,
            ui_name: str = None):
        """Generate a definition from a parsed enum
        """
        self.target_class = target_class
        self.unique_name = unique_name
        self.ui_name = ui_name or target_class.__name__
        self.module_name = module_name

        # begin building the ogn descriptor
        self.descriptor: Dict = {}
        self.descriptor["uiName"] = ui_name
        self.descriptor["version"] = 1
        self.descriptor["language"] = "Python"
        self.descriptor["description"] = f"Enum Wrapper for {self.ui_name}"
        self.descriptor["inputs"] = OrderedDict({
            "enum": {
                "uiName": "Input",
                "description": "Enum input",
                "type": "uint64",
                "default": 0,
                "metadata": {
                    "python_type_desc" : self.unique_name }},
            "exec": {
                "uiName": "Exec",
                "description": "Execution input",
                "type": "execution",
                "default": 0 }})

        def signature(name):
            return {
                "description": f"Execute on {name}",
                "type": "execution",
                "default": 0 }

        self.descriptor["outputs"] = OrderedDict({
            name: signature(name) for name in self.target_class.__members__ })

    # --------------------------------------------------------------------------------
    def get_unique_name(self) -> str:
        """overloaded from AutoNodeDefinitionWrapper"""
        return self.unique_name

    # --------------------------------------------------------------------------------
    def get_module_name(self) -> str:
        """overloaded from AutoNodeDefinitionWrapper"""
        return self.module_name

    # --------------------------------------------------------------------------------
    def get_node_impl(self):
        """overloaded from AutoNodeDefinitionWrapper"""
        class OgnEnumReturnType(
                OgnEnumExecutionWrapper,
                target_class=self.target_class):
            pass
        return OgnEnumReturnType

    # --------------------------------------------------------------------------------
    def get_ogn(self) -> Dict:
        """overloaded from AutoNodeDefinitionWrapper"""
        d = {self.unique_name: self.descriptor}
        return d


# ================================================================================
class EnumAutoNodeDefinitionGenerator(og.AutoNodeDefinitionGenerator):

    _name = "Enum"

    # --------------------------------------------------------------------------------
    @classmethod
    def generate_from_definitions(
            cls,
            target_type: type,
            type_name_sanitized: str,
            type_name_short: str,
            module_name: str) -> Tuple[Iterable[og.AutoNodeDefinitionWrapper], Iterable[str]]:

        members_covered = set()
        returned_generators = set()

        if hasattr(target_type, "__members__"):
            ret = OgnEnumWrapper(
                target_type,
                unique_name=type_name_sanitized,
                module_name=module_name,
                ui_name=f"Switch on {type_name_short}")

            members_covered.update(target_type.__members__)
            returned_generators.add(ret)

        return returned_generators, members_covered


# ================================================================================
#  submit this to AutoNode
og.register_autonode_type_extension(og.EnumAutoNodeDefinitionGenerator)

Implementation Details

This information is presented for debugging purposes. Do not rely on this information for your API.

Name Mangling

To avoid name collisions between two nodes from different origins, OGN mangles names of functions. If you are only using the UI to access your nodes, this shouldn’t interest you, but when accessing autogenerated nodes from code, the full node name is used.

Mangling is done according to these rules:

• Function specifiers have their . s replaced with __FUNC__ as an infix, like: MyClass.Function -> MyClass__FUNC__Function

• Property specifiers have their . s replaced with __PROP__ as an infix, like: MyClass.Property -> MyClass__PROP__Property

• Property getters and setters are suffixed as __GET and __SET, like so: MyClass__PROP__Property__GET and MyClass__PROP__Property__SET

• If any of those are nested inside another namespace will replace their separating dots with __NSP__

Here is an example with conversions:

import omni.graph.core as og

@og.AutoClass(module_name="test_module")
class TestClass:

        class TestSubclass:
            # TestClass__NSP__TestSubclass__PROP__public_var__GET
            # TestClass__NSP__TestSubclass__PROP__public_var__SET
            public_var: float = 3.141

    # TestClass__FUNC__public_method
    def public_method(self, exponent: float) -> float:
        return self.public_var ** exponent

This is done in order to spawn the node with code, like so:

import omni.graph.core as og

# This brings up `add_one` from the example above.
path_01 = '/Test/MyNodes/AddOne'
add_one = og.Controller.create_node(path_01, 'test_module.add_one')

# This brings up `TestClass.public_method` from the example above.
path_02 = '/Test/MyNodes/TestClassGetPublicMethod'
public_method = og.Controller.create_node(path_02, 'test_module.TestClass__FUNC__public_method')

path_03 = '/Test/MyNodes/TestClassGetPublicMethod'
public_var_getter = og.Controller.create_node(path_03, 'test_module.TestClass__NSP__TestSubclass__PROP__public_var__GET')