Package mdp :: Class Node

Class Node

A Node is the basic building block of an MDP application.

It represents a data processing element, like for example a learning algorithm, a data filter, or a visualization step. Each node can have one or more training phases, during which the internal structures are learned from training data (e.g. the weights of a neural network are adapted or the covariance matrix is estimated) and an execution phase, where new data can be processed forwards (by processing the data through the node) or backwards (by applying the inverse of the transformation computed by the node if defined).

Nodes have been designed to be applied to arbitrarily long sets of data: if the underlying algorithms supports it, the internal structures can be updated incrementally by sending multiple batches of data (this is equivalent to online learning if the chunks consists of single observations, or to batch learning if the whole data is sent in a single chunk). It is thus possible to perform computations on amounts of data that would not fit into memory or to generate data on-the-fly.

A Node also defines some utility methods, like for example copy and save, that return an exact copy of a node and save it in a file, respectively. Additional methods may be present, depending on the algorithm.

Node subclasses should take care of overwriting (if necessary) the functions is_trainable, _train, _stop_training, _execute, is_invertible, _inverse, _get_train_seq, and _get_supported_dtypes. If you need to overwrite the getters and setters of the node's properties refer to the docstring of get_input_dim/set_input_dim, get_output_dim/set_output_dim, and get_dtype/set_dtype.

Instance Methods

[hide private]

__add__(self, other)

__call__(self, x, *args, **kwargs)
Calling an instance of Node is equivalent to calling its execute method.

__init__(self, input_dim=None, output_dim=None, dtype=None)
If the input dimension and the output dimension are unspecified, they will be set when the train or execute method is called for the first time. If dtype is unspecified, it will be inherited from the data it receives at the first call of train or execute.

__repr__(self)
repr(x)

__str__(self)
str(x)

_check_input(self, x)

_check_output(self, y)

_check_train_args(self, x, *args, **kwargs)

_execute(self, x)

_get_supported_dtypes(self)
Return the list of dtypes supported by this node.

_get_train_seq(self)

_if_training_stop_training(self)

_inverse(self, x)

_pre_execution_checks(self, x)
This method contains all pre-execution checks.

_pre_inversion_checks(self, y)
This method contains all pre-inversion checks.

_refcast(self, x)
Helper function to cast arrays to the internal dtype.

_set_dtype(self, t)

_set_input_dim(self, n)

_set_output_dim(self, n)

_stop_training(self, *args, **kwargs)

_train(self, x)

copy(self, protocol=None)
Return a deep copy of the node.

execute(self, x, *args, **kwargs)
Process the data contained in x.

get_current_train_phase(self)
Return the index of the current training phase.

get_dtype(self)
Return dtype.

get_input_dim(self)
Return input dimensions.

get_output_dim(self)
Return output dimensions.

get_remaining_train_phase(self)
Return the number of training phases still to accomplish.

get_supported_dtypes(self)
Return dtypes supported by the node as a list of numpy.dtype objects.

has_multiple_training_phases(self)
Return True if the node has multiple training phases.

inverse(self, y, *args, **kwargs)
Invert y.

is_training(self)
Return True if the node is in the training phase, False otherwise.

save(self, filename, protocol=-1)
Save a pickled serialization of the node to filename. If filename is None, return a string.

set_dtype(self, t)
Set internal structures' dtype.

set_input_dim(self, n)
Set input dimensions.

set_output_dim(self, n)
Set output dimensions.

stop_training(self, *args, **kwargs)
Stop the training phase.

train(self, x, *args, **kwargs)
Update the internal structures according to the input data x.

Inherited from unreachable.newobject: __long__, __native__, __nonzero__, __unicode__, next

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __subclasshook__

Static Methods

[hide private]

is_invertible()
Return True if the node can be inverted, False otherwise.

is_trainable()
Return True if the node can be trained, False otherwise.

Properties

[hide private]

_train_seq
List of tuples:

dtype
dtype

input_dim
Input dimensions

output_dim
Output dimensions

supported_dtypes
Supported dtypes

Inherited from object: __class__

Method Details

[hide private]

add(self, other)
(Addition operator)

call(self, x, *args, **kwargs)
(Call operator)

Calling an instance of Node is equivalent to calling its execute method.

init(self, input_dim=None, output_dim=None, dtype=None)
(Constructor)

If the input dimension and the output dimension are unspecified, they will be set when the train or execute method is called for the first time. If dtype is unspecified, it will be inherited from the data it receives at the first call of train or execute.

Every subclass must take care of up- or down-casting the internal structures to match this argument (use _refcast private method when possible).

Overrides: object.__init__

repr(self)
(Representation operator)

repr(x)

Overrides: object.__repr__: (inherited documentation)

str(self)
(Informal representation operator)

str(x)

Overrides: object.__str__: (inherited documentation)

_check_input(self, x)

_check_output(self, y)

_check_train_args(self, x, *args, **kwargs)

_execute(self, x)

_get_supported_dtypes(self)

Return the list of dtypes supported by this node.

The types can be specified in any format allowed by numpy.dtype.

_get_train_seq(self)

_if_training_stop_training(self)

_inverse(self, x)

_pre_execution_checks(self, x)

This method contains all pre-execution checks.

It can be used when a subclass defines multiple execution methods.

_pre_inversion_checks(self, y)

This method contains all pre-inversion checks.

It can be used when a subclass defines multiple inversion methods.

_refcast(self, x)

Helper function to cast arrays to the internal dtype.

_set_dtype(self, t)

_set_input_dim(self, n)

_set_output_dim(self, n)

_stop_training(self, *args, **kwargs)

_train(self, x)

copy(self, protocol=None)

Return a deep copy of the node.

Parameters:

protocol - the pickle protocol (deprecated).

execute(self, x, *args, **kwargs)

Process the data contained in x.

If the object is still in the training phase, the function stop_training will be called. x is a matrix having different variables on different columns and observations on the rows.

By default, subclasses should overwrite _execute to implement their execution phase. The docstring of the _execute method overwrites this docstring.

get_current_train_phase(self)

Return the index of the current training phase.

The training phases are defined in the list self._train_seq.

get_dtype(self)

Return dtype.

get_input_dim(self)

Return input dimensions.

get_output_dim(self)

Return output dimensions.

get_remaining_train_phase(self)

Return the number of training phases still to accomplish.

If the node is not trainable then return 0.

get_supported_dtypes(self)

Return dtypes supported by the node as a list of numpy.dtype objects.

Note that subclasses should overwrite self._get_supported_dtypes when needed.

has_multiple_training_phases(self)

Return True if the node has multiple training phases.

inverse(self, y, *args, **kwargs)

Invert y.

If the node is invertible, compute the input x such that y = execute(x).

By default, subclasses should overwrite _inverse to implement their inverse function. The docstring of the inverse method overwrites this docstring.

is_invertible()
Static Method

Return True if the node can be inverted, False otherwise.

is_trainable()
Static Method

Return True if the node can be trained, False otherwise.

is_training(self)

Return True if the node is in the training phase, False otherwise.

save(self, filename, protocol=-1)

Save a pickled serialization of the node to filename. If filename is None, return a string.

Note: the pickled Node is not guaranteed to be forwards or backwards compatible.

set_dtype(self, t)

Set internal structures' dtype.

Perform sanity checks and then calls self._set_dtype(n), which is responsible for setting the internal attribute self._dtype. Note that subclasses should overwrite self._set_dtype when needed.

set_input_dim(self, n)

Set input dimensions.

Perform sanity checks and then calls self._set_input_dim(n), which is responsible for setting the internal attribute self._input_dim. Note that subclasses should overwrite self._set_input_dim when needed.

set_output_dim(self, n)

Set output dimensions.

Perform sanity checks and then calls self._set_output_dim(n), which is responsible for setting the internal attribute self._output_dim. Note that subclasses should overwrite self._set_output_dim when needed.

stop_training(self, *args, **kwargs)

Stop the training phase.

By default, subclasses should overwrite _stop_training to implement this functionality. The docstring of the _stop_training method overwrites this docstring.

train(self, x, *args, **kwargs)

Update the internal structures according to the input data x.

x is a matrix having different variables on different columns and observations on the rows.

By default, subclasses should overwrite _train to implement their training phase. The docstring of the _train method overwrites this docstring.

Note: a subclass supporting multiple training phases should implement the same signature for all the training phases and document the meaning of the arguments in the _train method doc-string. Having consistent signatures is a requirement to use the node in a flow.

Property Details

[hide private]

_train_seq

List of tuples:

[(training-phase1, stop-training-phase1),
 (training-phase2, stop_training-phase2),
 ...]

By default:

_train_seq = [(self._train, self._stop_training)]

Get Method:: unreachable(self)

dtype

Get Method:: get_dtype(self) - Return dtype.
Set Method:: set_dtype(self, t) - Set internal structures' dtype.

input_dim

Input dimensions

Get Method:: get_input_dim(self) - Return input dimensions.
Set Method:: set_input_dim(self, n) - Set input dimensions.

output_dim

Output dimensions

Get Method:: get_output_dim(self) - Return output dimensions.
Set Method:: set_output_dim(self, n) - Set output dimensions.

supported_dtypes

Supported dtypes

Get Method:: get_supported_dtypes(self) - Return dtypes supported by the node as a list of numpy.dtype objects.

Class Node

__add__(self, other) (Addition operator)

__call__(self, x, *args, **kwargs) (Call operator)

__init__(self, input_dim=None, output_dim=None, dtype=None) (Constructor)

__repr__(self) (Representation operator)

__str__(self) (Informal representation operator)

_check_input(self, x)

_check_output(self, y)

_check_train_args(self, x, *args, **kwargs)

_execute(self, x)

_get_supported_dtypes(self)

_get_train_seq(self)

_if_training_stop_training(self)

_inverse(self, x)

_pre_execution_checks(self, x)

_pre_inversion_checks(self, y)

_refcast(self, x)

_set_dtype(self, t)

_set_input_dim(self, n)

_set_output_dim(self, n)

_stop_training(self, *args, **kwargs)

_train(self, x)

copy(self, protocol=None)

execute(self, x, *args, **kwargs)

get_current_train_phase(self)

get_dtype(self)

get_input_dim(self)

get_output_dim(self)

get_remaining_train_phase(self)

get_supported_dtypes(self)

has_multiple_training_phases(self)

inverse(self, y, *args, **kwargs)

is_invertible() Static Method

is_trainable() Static Method

is_training(self)

save(self, filename, protocol=-1)

set_dtype(self, t)

set_input_dim(self, n)

set_output_dim(self, n)

stop_training(self, *args, **kwargs)

train(self, x, *args, **kwargs)

_train_seq

dtype

input_dim

output_dim

supported_dtypes

add(self, other)
(Addition operator)

call(self, x, *args, **kwargs)
(Call operator)

init(self, input_dim=None, output_dim=None, dtype=None)
(Constructor)

repr(self)
(Representation operator)

str(self)
(Informal representation operator)

is_invertible()
Static Method

is_trainable()
Static Method