Package mdp :: Package parallel :: Class ParallelCheckpointFlow

Class ParallelCheckpointFlow

Parallel version of CheckpointFlow.

Note that train phases are always closed, so e.g. CheckpointSaveFunction
should not expect open train phases. This is necessary since otherwise
stop_training() would be called remotely.

Instance Methods

[hide private]

__init__(self, flow, verbose=False, **kwargs)
Initialize the internal variables.

_post_stop_training_hook(self)
Check if we reached a checkpoint.

setup_parallel_training(self, data_iterables, checkpoints, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, **kwargs)
Checkpoint version of parallel training.

train(self, data_iterables, checkpoints, scheduler=None, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, overwrite_result_container=True, **kwargs)
Train all trainable nodes in the flow.

Inherited from unreachable.newobject: __long__, __native__, __nonzero__, __unicode__, next

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __subclasshook__

Inherited from ParallelFlow

_create_execute_task(self)
Create and return a single execution task.

_create_train_task(self)
Create and return a single training task without callable.

_local_train_phase(self, data_iterable)
Perform a single training phase locally.

_next_train_phase(self)
Find the next phase or node for parallel training.

execute(self, iterable, nodenr=None, scheduler=None, execute_callable_class=None, overwrite_result_container=True)
Train all trainable nodes in the flow.

get_task(self)
Return a task either for either training or execution.

setup_parallel_execution(self, iterable, nodenr=None, execute_callable_class=<class 'mdp.parallel.FlowExecuteCallable'>)
Prepare the flow for handing out tasks to do the execution.

use_results(self, results)
Use the result from the scheduler.

Inherited from CheckpointFlow

_train_check_checkpoints(self, checkpoints)

Inherited from Flow

__add__(self, other)

__call__(self, iterable, nodenr=None)
Calling an instance is equivalent to call its 'execute' method.

__contains__(self, item)

__delitem__(self, key)

__getitem__(self, key)

__iadd__(self, other)

__iter__(self)

__len__(self)

__repr__(self)
repr(x)

__setitem__(self, key, value)

__str__(self)
str(x)

_check_dimension_consistency(self, out, inp)
Raise ValueError when both dimensions are set and different.

_check_nodes_consistency(self, flow=None)
Check the dimension consistency of a list of nodes.

_check_value_type_isnode(self, value)

_close_last_node(self)

_execute_seq(self, x, nodenr=None)

_inverse_seq(self, x)

_propagate_exception(self, except_, nodenr)

_stop_training_hook(self)
Hook method that is called before stop_training is called.

_train_check_iterables(self, data_iterables)
Return the data iterables after some checks and sanitizing.

_train_node(self, data_iterable, nodenr)
Train a single node in the flow.

append(flow, node)
append node to flow end

copy(self, protocol=None)
Return a deep copy of the flow.

extend(flow, iterable)
extend flow by appending elements from the iterable

insert(flow, index, node)
insert node before index

inverse(self, iterable)
Process the data through all nodes in the flow backwards (starting from the last node up to the first node) by calling the inverse function of each node. Of course, all nodes in the flow must be invertible.

node

pop(flow, index=...)
remove and return node at index (default last)

save(self, filename, protocol=-1)
Save a pickled serialization of the flow to 'filename'. If 'filename' is None, return a string.

set_crash_recovery(self, state=True)
Set crash recovery capabilities.

Static Methods

[hide private]

Inherited from Flow

_get_required_train_args(node)
Return arguments in addition to self and x for node.train.

Properties

[hide private]

Inherited from object: __class__

Inherited from ParallelFlow

is_parallel_executing
Return True if parallel execution is underway.

is_parallel_training
Return True if parallel training is underway.

task_available
Return True if tasks are available, otherwise False.

Method Details

[hide private]

init(self, flow, verbose=False, **kwargs)
(Constructor)

Initialize the internal variables.

Overrides: object.__init__

_post_stop_training_hook(self)

Check if we reached a checkpoint.

Overrides: ParallelFlow._post_stop_training_hook

setup_parallel_training(self, data_iterables, checkpoints, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, **kwargs)

Checkpoint version of parallel training.

Overrides: ParallelFlow.setup_parallel_training

train(self, data_iterables, checkpoints, scheduler=None, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, overwrite_result_container=True, **kwargs)

Train all trainable nodes in the flow.

Same as the train method in ParallelFlow, but with additional support
of checkpoint functions as in CheckpointFlow.

Overrides: Flow.train

Class ParallelCheckpointFlow

__init__(self, flow, verbose=False, **kwargs) (Constructor)

_post_stop_training_hook(self)

setup_parallel_training(self, data_iterables, checkpoints, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, **kwargs)

train(self, data_iterables, checkpoints, scheduler=None, train_callable_class=<class 'mdp.parallel.FlowTrainCallable'>, overwrite_result_container=True, **kwargs)

init(self, flow, verbose=False, **kwargs)
(Constructor)