Deciding What To Expose As The Python API In OmniGraph

  • Status: proposed

  • Deciders: kpicott, OmniGraph devs

  • Date: 2022-07-07

Technical Story: OM-46154

Context and Problem Statement

Once the Python API definition was decided there was the further decision of exactly what should go into the public API, what should be internal, and what should be completely deprecated.

Considered Options

  1. The Wild West - everything that was already exposed continues to be exposed in exactly the same way

  2. Complete Lockdown - only what is strictly necessary for current functionality to work is exposed, everything else is made inaccessible to the user

  3. Graceful Deprecation - we define what we want exposed and make those the __all__ exposed symbols, but we also include all of the previously available symbols in the module itself. Soft deprecation warnings are put in for anything we want to eventually completely hide from the user.

Decision Outcome

Option 3: the users get full compatibility without any code changes but we have drawn our line in the sand denoting exactly what we will be supporting in the future.

The exposure of symbols will be broken into different categories depending on the exposure desired: 1. Full Exposure the module imports the symbol and its name is added to the module’s __all__ list 2. Internal Exposure the symbol is renamed to have a leading underscore and the module imports the symbol. The name of the symbol is not added to the module’s __all__ list 3. Hidden Exposure for symbols that should be internal but for which the work to rename was too involved for now, create a new exposure list called _HIDDEN in the same location where __all__ is defined and but all of the symbols that should be made internal into that list. Eventually this list will be empty as symbols are renamed to reflect the fact that they are only internal to OmniGraph 4. Soft Deprecation for symbols we no longer want to support. Their import and/or definition will be moved to a module file with the current version number, e.g. _1_11.py. The deprecated symbols will be added to that module’s __all__ list and the module will be imported into the main module but not added to the main module’s __all__ list 5. Hard Deprecation for symbols that were deprecated before that can simply be deleted. The file from which they were imported is replaced by a file that is empty except for a single raise DeprecationError statement explaining what action the user has to take to replace the deprecated functionality

To reduce the size of the largest modules, submodules will be created to segment by grouped functionality. For example, omni.graph.autonode will contain all of the AutoNode-specific functionality.

Lastly, in order to support this model some reorganization of files will be necessary. In particular, anything that is not part of the public API should be moved to an _impl subdirectory with imports from it being adjusted to match.

Pros and Cons of the Options

Option 1

  • Good - work required to define the API is minimal as it is just what we have

  • Bad - with the existing structure it’s difficult to ascertain exactly what the API is, and as a consequence it would be harder to maintain in the future

  • Ugly - the current surface is organic anarchy; continuing to support all of it would be a development drag

Option 2

  • Good - with everything well defined we make a strong statement as to what we are claiming to support.

  • Bad - we would have to rely on tests to find any missing exports, and we know we don’t have 100% coverage even within Kit

  • Ugly - we are already at the stage where we don’t know for sure what users are using, and the testing matrix is not yet populated enough to give us any confidence we will find the problems so there as an almost certainty that we would end up breaking somebody’s code

Option 3

  • Good - users’s are happy because they have no immediate work to do, and we are happy because we have defined exactly what we are claiming to support. Because it’s a smaller subset we can write a compatibility test for future changes.

  • Bad - we will end up exposing some things we don’t want to, simply because it’s too difficult to remove in one step

  • Ugly - everything is still wide open and users are free to make use of anything in our system. Despite any warnings to the contrary Hyrum’s Law dictates that it will be used anyway.