Package mdp :: Package nodes :: Class RBMWithLabelsNode

Class RBMWithLabelsNode

Restricted Boltzmann Machine with softmax labels. An RBM is an undirected probabilistic network with binary variables. In this case, the node is partitioned into a set of observed (visible) variables, a set of hidden (latent) variables, and a set of label variables (also observed), only one of which is active at any time. The node is able to learn associations between the visible variables and the labels. By default, the execute method returns the probability of one of the hiden variables being equal to 1 given the input. Use the sample_v method to sample from the observed variables (visible and labels) given a setting of the hidden variables, and sample_h to do the opposite. The energy method can be used to compute the energy of a given setting of all variables.

Reference

The network is trained by Contrastive Divergence, as described in Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1711-1800

For more information on RBMs with labels, see:

Geoffrey E. Hinton (2007) Boltzmann machine. Scholarpedia, 2(5):1668.
Hinton, G. E, Osindero, S., and Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554.

Instance Methods

[hide private]

__init__(self, hidden_dim, labels_dim, visible_dim=None, dtype=None)
Initializes an object of type 'RBMWithLabelsNode'.

_sample_v(self, h, sample_l=False, concatenate=True)

_set_input_dim(self, n)

float

energy(self, v, h, l)
Compute the energy of the RBM given observed variables state v and l, and hidden variables state h.

float

execute(self, v, l, return_probs=True)
If return_probs is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:] and l[n,:].

tuple

sample_h(self, v, l)
Sample the hidden variables given observations v and labels l.

tuple

sample_v(self, h)
Sample the observed variables given hidden variable state h.

train(self, v, l, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)
Update the internal structures according to the visible data v and the labels l. The training is performed using Contrastive Divergence (CD).

Inherited from unreachable.newobject: __long__, __native__, __nonzero__, __unicode__, next

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __subclasshook__

Inherited from RBMNode

_energy(self, v, h)

float

_execute(self, v, return_probs=True)
If return_probs is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:]. If return_probs is False, return a sample from that probability.

_init_weights(self)

_pre_inversion_checks(self, y)
This method contains all pre-inversion checks.

_sample_h(self, v)

_stop_training(self)

_train(self, v, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, update_with_ph=True, verbose=False)
Update the internal structures according to the input data v. The training is performed using Contrastive Divergence (CD).

stop_training(self)
Stop the training phase.

Inherited from Node

__add__(self, other)

__call__(self, x, *args, **kwargs)
Calling an instance of Node is equivalent to calling its execute method.

__repr__(self)
repr(x)

__str__(self)
str(x)

_check_input(self, x)

_check_output(self, y)

_check_train_args(self, x, *args, **kwargs)

_get_supported_dtypes(self)
Return the list of dtypes supported by this node.

_get_train_seq(self)

_if_training_stop_training(self)

_inverse(self, x)

_pre_execution_checks(self, x)
This method contains all pre-execution checks.

_refcast(self, x)
Helper function to cast arrays to the internal dtype.

_set_dtype(self, t)

_set_output_dim(self, n)

copy(self, protocol=None)
Return a deep copy of the node.

get_current_train_phase(self)
Return the index of the current training phase.

get_dtype(self)
Return dtype.

get_input_dim(self)
Return input dimensions.

get_output_dim(self)
Return output dimensions.

get_remaining_train_phase(self)
Return the number of training phases still to accomplish.

get_supported_dtypes(self)
Return dtypes supported by the node as a list of numpy.dtype objects.

has_multiple_training_phases(self)
Return True if the node has multiple training phases.

inverse(self, y, *args, **kwargs)
Invert y.

is_training(self)
Return True if the node is in the training phase, False otherwise.

save(self, filename, protocol=-1)
Save a pickled serialization of the node to filename. If filename is None, return a string.

set_dtype(self, t)
Set internal structures' dtype.

set_input_dim(self, n)
Set input dimensions.

set_output_dim(self, n)
Set output dimensions.

Static Methods

[hide private]

is_invertible()
Return True if the node can be inverted, False otherwise.

Inherited from Node

is_trainable()
Return True if the node can be trained, False otherwise.

Instance Variables

[hide private]

bh
Bias vector of the hidden variables.

bv
Bias vector of the observed variables.

w
Generative weights between hidden and observed variables.

Properties

[hide private]

Inherited from object: __class__

Inherited from Node

_train_seq
List of tuples:

dtype
dtype

input_dim
Input dimensions

output_dim
Output dimensions

supported_dtypes
Supported dtypes

Method Details

[hide private]

init(self, hidden_dim, labels_dim, visible_dim=None, dtype=None)
(Constructor)

Initializes an object of type 'RBMWithLabelsNode'.

Parameters:

hidden_dim (int) - Number of hidden variables.
labels_dim (int) - Number of labels.
visible_dim (int) - Number of observed variables. Default is None.
dtype (numpy.dtype, str) - Datatype of the input. Default is None.

Overrides: object.__init__

_sample_v(self, h, sample_l=False, concatenate=True)

Overrides: RBMNode._sample_v

_set_input_dim(self, n)

Overrides: Node._set_input_dim

energy(self, v, h, l)

Compute the energy of the RBM given observed variables state v and l, and hidden variables state h.

Parameters:

v (numpy.ndarray) - A binary matrix having different variables on different columns and observations on the rows.
l (numpy.ndarray) - The labels. A binary matrix having different variables on different columns and observations on the rows. Only one value per row should be 1.
h (numpy.ndarray) - The hidden variable state h.

Returns: float

The energy of the RBM given observed and the hidden variables.

Overrides: RBMNode.energy

execute(self, v, l, return_probs=True)

If return_probs is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:] and l[n,:].

If return_probs is False, return a sample from that probability.

Parameters:

v (numpy.ndarray) - A binary matrix having different variables on different columns and observations on the rows.
l (numpy.ndarray) - The labels. A binary matrix having different variables on different columns and observations on the rows. Only one value per row should be 1.
return_probs (bool) - Controls the return value. Default value: True

Returns: float

The probability of the hidden variables being 1 given the observations and labels or a sample from that probability.

Overrides: Node.execute

is_invertible()
Static Method

Return True if the node can be inverted, False otherwise.

Overrides: Node.is_invertible: (inherited documentation)

sample_h(self, v, l)

Sample the hidden variables given observations v and labels l.

Parameters:

v (numpy.ndarray) - A binary matrix having different variables on different columns and observations on the rows.
l (numpy.ndarray) - The labels. A binary matrix having different variables on different columns and observations on the rows. Only one value per row should be 1.

Returns: tuple

A tuple (prob_h, h), where prob_h[n,i] is the probability that variable i is one given the observations v[n,:] and the labels l[n,:], and h[n,i] is a sample from the posterior probability.

Overrides: RBMNode.sample_h

sample_v(self, h)

Sample the observed variables given hidden variable state h.

Parameters:

h (numpy.ndarray) - The hidden variable state h.

Returns: tuple

A tuple (prob_v, probs_l, v, l), where prob_v[n,i] is the probability that the visible variable i is one given the hidden variables h[n,:], and v[n,i] is a sample from that conditional probability. prob_l and l have similar interpretations for the label variables. Note that the labels are activated using a softmax function, so that only one label can be active at any time.

Overrides: RBMNode.sample_v

train(self, v, l, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)

Update the internal structures according to the visible data v and the labels l. The training is performed using Contrastive Divergence (CD).

Parameters:

v (numpy.ndarray) - A binary matrix having different variables on different columns and observations on the rows.
l (numpy.ndarray) - A binary matrix having different variables on different columns and observations on the rows. Only one value per row should be 1.
n_updates (int) - Number of CD iterations. Default value: 1
epsilon (float) - Learning rate. Default value: 0.1
decay (float) - Weight decay term. Default value: 0.
momentum (float) - Momentum term. Default value: 0.
verbose (bool) - Controls the verbosity.

Overrides: Node.train

Instance Variable Details

[hide private]

bh

Bias vector of the hidden variables.

bv

Bias vector of the observed variables.

w

Generative weights between hidden and observed variables.

Class RBMWithLabelsNode

__init__(self, hidden_dim, labels_dim, visible_dim=None, dtype=None) (Constructor)

_sample_v(self, h, sample_l=False, concatenate=True)

_set_input_dim(self, n)

energy(self, v, h, l)

execute(self, v, l, return_probs=True)

is_invertible() Static Method

sample_h(self, v, l)

sample_v(self, h)

train(self, v, l, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)

bh

bv

w

init(self, hidden_dim, labels_dim, visible_dim=None, dtype=None)
(Constructor)

is_invertible()
Static Method