plugins.h_tqdm.ProgressBarΒΆ
Provides a progress bar for Hamilton execution. Must have tqdm installed to use it:
pip install sf-hamilton[tqdm] (use quotes if using zsh)
- class hamilton.plugins.h_tqdm.ProgressBar(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)ΒΆ
An adapter that uses tqdm to show progress bars for the graph execution.
Note: you need to have tqdm installed for this to work. If you donβt have it installed, you can install it with pip install tqdm (or pip install sf-hamilton[tqdm] β use quotes if youβre using zsh).
from hamilton.plugins import h_tqdm dr = ( driver.Builder() .with_config({}) .with_modules(some_modules) .with_adapters(h_tqdm.ProgressBar(desc="DAG-NAME")) .build() ) # and then when you call .execute() or .materialize() you'll get a progress bar!
- __init__(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)ΒΆ
Create a new Progress Bar adapter.
- Parameters:
desc β The description to show in the progress bar. E.g. DAG Name is a good choice.
kwargs β Additional kwargs to pass to TQDM. See TQDM docs for more info.
node_name_target_width β the target width for the node name so that the progress bar is consistent. If this is None, it will take the longest, until it hits max_node_name_width.
- post_graph_execute(*, run_id: str, graph: FunctionGraph, success: bool, error: Exception | None, results: Dict[str, Any] | None)ΒΆ
Just delegates to the interface method, passing in the right data.
- post_node_execute(*, run_id: str, node_: Node, kwargs: Dict[str, Any], success: bool, error: Exception | None, result: Any | None, task_id: str | None = None)ΒΆ
Wraps the after_execution method, providing a bridge to an external-facing API. Do not override this!
- pre_graph_execute(*, run_id: str, graph: FunctionGraph, final_vars: List[str], inputs: Dict[str, Any], overrides: Dict[str, Any])ΒΆ
Implementation of the pre_graph_execute hook. This just converts the inputs to the format the user-facing hook is expecting β performing a walk of the DAG to pass in the set of nodes to execute. Delegates to the interface method.
- pre_node_execute(*, run_id: str, node_: Node, kwargs: Dict[str, Any], task_id: str | None = None)ΒΆ
Wraps the before_execution method, providing a bridge to an external-facing API. Do not override this!
- run_after_graph_execution(**future_kwargs)ΒΆ
This is run after graph execution. This allows you to do anything you want after the graph executes, knowing the results of the execution/any errors.
- Parameters:
graph β Graph that is being executed
results β Results of the graph execution
error β Error that occurred, None if no error occurred
success β Whether the graph executed successfully
run_id β Run ID (unique in process scope) of the current run. Use this to track state.
future_kwargs β Additional keyword arguments β this is kept for backwards compatibility
- run_after_node_execution(**future_kwargs)ΒΆ
Hook that is executed post node execution.
- Parameters:
node_name β Name of the node in question
node_tags β Tags of the node
node_kwargs β Keyword arguments passed to the node
node_return_type β Return type of the node
result β Output of the node, None if an error occurred
error β Error that occurred, None if no error occurred
success β Whether the node executed successfully
task_id β The ID of the task, none if not in a task-based environment
run_id β Run ID (unique in process scope) of the current run. Use this to track state.
future_kwargs β Additional keyword arguments β this is kept for backwards compatibility
- run_before_graph_execution(*, graph: HamiltonGraph, final_vars: List[str], inputs: Dict[str, Any], overrides: Dict[str, Any], execution_path: Collection[str], **future_kwargs: Any)ΒΆ
This is run prior to graph execution. This allows you to do anything you want before the graph executes, knowing the basic information that was passed in.
- Parameters:
graph β Graph that is being executed
final_vars β Output variables of the graph
inputs β Input variables passed to the graph
overrides β Overrides passed to the graph
execution_path β Collection of nodes that will be executed β these are just the nodes (not input nodes) that will be run during the course of execution.
run_id β Run ID (unique in process scope) of the current run. Use this to track state.
future_kwargs β Additional keyword arguments β this is kept for backwards compatibility
- run_before_node_execution(*, node_name: str, node_tags: Dict[str, Any], node_kwargs: Dict[str, Any], node_return_type: type, task_id: str | None, **future_kwargs: Any)ΒΆ
Hook that is executed prior to node execution.
- Parameters:
node_name β Name of the node.
node_tags β Tags of the node
node_kwargs β Keyword arguments to pass to the node
node_return_type β Return type of the node
task_id β The ID of the task, none if not in a task-based environment
run_id β Run ID (unique in process scope) of the current run. Use this to track state.
node_input_types β the input types to the node and what it is expecting
future_kwargs β Additional keyword arguments β this is kept for backwards compatibility