3. Plotting and Debugging

Plotting

For Errors & debugging it is necessary to visualize the graph-operation. You may plot the original plot and annotate on top the execution plan and solution of the last computation, calling methods with arguments like this:

netop.plot(show=True)              # open a matplotlib window
netop.plot("netop.svg")            # other supported formats: png, jpg, pdf, ...
netop.plot()                       # without arguments return a pydot.DOT object
netop.plot(solution=solution)      # annotate graph with solution values

… or for the last …:

solution.plot(...)
execution plan
Graphtik Legend

The legend for all graphtik diagrams, generated by legend().

The same Plotter.plot() method applies also for:

each one capable to producing diagrams with increasing complexity. Whenever possible, the top-level plot() methods will delegate to the ones below; specifically, the netop keeps a transient reference to the last plan. BUT the plan does not hold such a reference, you have to plot the solution.

For instance, when a net-operation has just been composed, plotting it will come out bare bone, with just the 2 types of nodes (data & operations), their dependencies, and the sequence of the execution-plan.

barebone graph

But as soon as you run it, the net plot calls will print more of the internals. Internally it delegates to ExecutionPlan.plot() of NetworkOperation.last_plan attribute, which caches the last run to facilitate debugging. If you want the bare-bone diagram, plot the network:

netop.net.plot(...)

If you want all details, plot the solution:

solution.net.plot(...)

Note

For plots, Graphviz program must be in your PATH, and pydot & matplotlib python packages installed. You may install both when installing graphtik with its plot extras:

pip install graphtik[plot]

Tip

The pydot.Dot instances returned by Plotter.plot() are rendered directly in Jupyter/IPython notebooks as SVG images.

You may increase the height of the SVG cell output with something like this:

netop.plot(jupyter_render={"svg_element_styles": "height: 600px; width: 100%"})

Check default_jupyter_render for defaults.

Errors & debugging

Graphs may become arbitrary deep. Launching a debugger-session to inspect deeply nested stacks is notoriously hard

As a workaround, when some operation fails, the original exception gets annotated with the folllowing properties, as a debug aid:

>>> from graphtik import compose, operation
>>> from pprint import pprint
>>> def scream(*args):
...     raise ValueError("Wrong!")
>>> try:
...     compose("errgraph",
...             operation(name="screamer", needs=['a'], provides=["foo"])(scream)
...     )(a=None)
... except ValueError as ex:
...     pprint(ex.jetsam)
{'aliases': None,
 'args': {'args': [None], 'kwargs': {}},
 'network': Network(
    +--a
    +--FunctionalOperation(name='screamer', needs=['a'], provides=['foo'], fn='scream')
    +--foo),
 'operation': FunctionalOperation(name='screamer', needs=['a'], provides=['foo'], fn='scream'),
 'outputs': None,
 'plan': ExecutionPlan(needs=['a'], provides=['foo'], x1 steps:
  +--FunctionalOperation(name='screamer', needs=['a'], provides=['foo'], fn='scream')),
 'provides': None,
 'results_fn': None,
 'results_op': None,
 'solution': {'a': None}}

In interactive REPL console you may use this to get the last raised exception:

import sys

sys.last_value.jetsam

The following annotated attributes might have meaningfull value on an exception:

network
the innermost network owning the failed operation/function
plan
the innermost plan that executing when a operation crashed
operation
the innermost operation that failed
args
either the input arguments list fed into the function, or a dict with both args & kwargs keys in it.
outputs
the names of the outputs the function was expected to return
provides
the names eventually the graph needed from the operation; a subset of the above, and not always what has been declared in the operation.
fn_results
the raw results of the operation’s fuction, if any
op_results
the results, always a dictionary, as matched with operation’s provides
solution
an instance of Solution, contains inputs & outputs till the error happened; note that Solution.executed contain the list of executed operations so far.

Ofcourse you may use many of the above “jetsam” values when plotting.

Note

The Plotting capabilities, along with the above annotation of exceptions with the internal state of plan/operation often renders a debugger session unnecessary. But since the state of the annotated values might be incomple, you may not always avoid one.

Execution internals

Compile & execute network graphs of operations.