theanolm.backend package

Submodules

theanolm.backend.classdistribution module

A module that implements classes for sampling noise words.

class theanolm.backend.classdistribution.ClassDistribution(random)

Bases: object

Base Class for Probability Distributions

A probability distribution class implements methods for sampling words and converting words to probabilities.

__abstractmethods__ = frozenset({'probs', 'sample'})

__dict__ = mappingproxy({'__module__': 'theanolm.backend.classdistribution', '__doc__': 'Base Class for Probability Distributions\n\n    A probability distribution class implements methods for sampling words and\n    converting words to probabilities.\n    ', '__init__': <function ClassDistribution.__init__>, 'sample': <function ClassDistribution.sample>, 'probs': <function ClassDistribution.probs>, '__dict__': <attribute '__dict__' of 'ClassDistribution' objects>, '__weakref__': <attribute '__weakref__' of 'ClassDistribution' objects>, '__abstractmethods__': frozenset({'probs', 'sample'}), '_abc_impl': <_abc._abc_data object>, '__annotations__': {}})

__init__(random)

Constructs a probability distribution.

Parameters:

random (MRG_RandomStreams) – a random number generator

__module__ = 'theanolm.backend.classdistribution'

__weakref__

list of weak references to the object (if defined)

_abc_impl = <_abc._abc_data object>

abstract probs(class_ids)

Converts class IDs to probabilities.

The returned value may be a NumPy array or Theano tensor. It may be an array of the same shape as class_ids or a scalar which can be broadcasted to that shape.

Parameters:

class_ids (numpy.ndarray) – classes whose probabilities are requested

Return type:

theano.config.floatX, numpy.ndarray, or Variable

Returns:

an array of the same shape as class_ids or a scalar which can be broadcasted to that shape

abstract sample(minibatch_size, num_samples)

Samples given number of class IDs per mini-batch element.

class theanolm.backend.classdistribution.LogUniformDistribution(random, support)

Bases: ClassDistribution

__abstractmethods__ = frozenset({})

__annotations__ = {}

__init__(random, support)

Constructs a probability distribution such that the logarithm of the samples is uniformly distributed.

Parameters:

• random (MRG_RandomStreams) – a random number generator

• support (int) – the sampled values will be in the range from 0 to support - 1

__module__ = 'theanolm.backend.classdistribution'

_abc_impl = <_abc._abc_data object>

probs(class_ids)

Converts class IDs to probabilities.

The returned value is a scalar. It will be broadcasted to the correct shape by Theano.

Parameters:

class_ids (Variable) – a symbolic vector describing the classes whose probabilities are requested

Return type:

Variable

Returns:

probabilities of the classes

sample(minibatch_size, num_samples)

Samples given number of class IDs per mini-batch element.

Parameters:

• minibatch_size (int) – number of mini-batch elements

• num_samples (int) – number of samples to draw for each mini-batch element

Return type:

Variable

Returns:

a 2-dimensional tensor variable describing the num_samples random samples for minibatch_size mini-batch element

class theanolm.backend.classdistribution.MultinomialDistribution(random, probs)

Bases: ClassDistribution

__abstractmethods__ = frozenset({})

__annotations__ = {}

__init__(random, probs)

Constructs a multinomial probability distribution.

Parameters:

• random (MRG_RandomStreams) – a random number generator

• probs (Variable) – a tensor vector that defines the distribution where to sample from

__module__ = 'theanolm.backend.classdistribution'

_abc_impl = <_abc._abc_data object>

probs(class_ids)

Converts class IDs to probabilities.

The returned value is a scalar. It will be broadcasted to the correct shape by Theano.

Parameters:

class_ids (Variable) – a symbolic vector describing the classes whose probabilities are requested

Return type:

Variable

Returns:

symbolic vector describing the probabilities of the classes

sample(minibatch_size, num_samples)

Samples given number of class IDs per mini-batch element.

Theano supports currently only sampling without replacement. Thus if a different set of samples is required for each mini-batch element, we repeat the distribution for each mini-batch element.

At some point the old interface, multinomial_wo_replacement() was faster, but is not supported anymore. In the future target='cpu' may improve the speed.

Parameters:

• minibatch_size (int) – number of mini-batch elements

• num_samples (int) – number of samples to draw for each mini-batch element

Return type:

Variable

Returns:

a 2-dimensional tensor variable describing the num_samples random samples for minibatch_size mini-batch element

class theanolm.backend.classdistribution.UniformDistribution(random, support)

Bases: ClassDistribution

__abstractmethods__ = frozenset({})

__annotations__ = {}

__init__(random, support)

Constructs a uniform probability distribution.

Parameters:

• random (MRG_RandomStreams) – a random number generator

• support (int) – the sampled values will be in the range from 0 to support - 1

__module__ = 'theanolm.backend.classdistribution'

_abc_impl = <_abc._abc_data object>

probs(class_ids)

Converts class IDs to probabilities.

The returned value is a scalar. It will be broadcasted to the correct shape by Theano.

Parameters:

class_ids (Variable) – a symbolic vector describing the classes whose probabilities are requested (ignored)

Return type:

theano.config.floatX

Returns:

the probability of a single class

sample(minibatch_size, num_samples)

Samples given number of class IDs per mini-batch element.

Parameters:

• minibatch_size (int) – number of mini-batch elements

• num_samples (int) – number of samples to draw for each mini-batch element

Return type:

Variable

Returns:

a 2-dimensional tensor variable describing the num_samples random samples for minibatch_size mini-batch element

theanolm.backend.debugfunctions module

theanolm.backend.exceptions module

A module that implements custom exception classes.

exception theanolm.backend.exceptions.IncompatibleStateError

Bases: Exception

Exception raised when attempting to load a state that is incompatible with the neural network structure.

__module__ = 'theanolm.backend.exceptions'

__weakref__

list of weak references to the object (if defined)

exception theanolm.backend.exceptions.InputError

Bases: Exception

Exception raised for errors in the input.

__module__ = 'theanolm.backend.exceptions'

__weakref__

list of weak references to the object (if defined)

exception theanolm.backend.exceptions.NumberError

Bases: Exception

Exception raised when one of the parameter gets NaN value.

__module__ = 'theanolm.backend.exceptions'

__weakref__

list of weak references to the object (if defined)

exception theanolm.backend.exceptions.TheanoConfigurationError

Bases: Exception

Exception raised when Theano has not been properly configured.

__module__ = 'theanolm.backend.exceptions'

__weakref__

list of weak references to the object (if defined)

theanolm.backend.filetypes module

A module that defines file type classes to be used with command-line argument parser.

class theanolm.backend.filetypes.BinaryFileType(mode='r')

Bases: object

An object that can be passed as the “type” argument to ArgumentParser.add_argument() in order to convert a path argument to a file object.

If the path ends in “.gz”, the file will be opened using gzip.open(). UTF-8 encoding will be assumed. The special path “-” means standard input or output.

Keyword Arguments:

• mode – A string indicating how the file is to be opened. Accepts the same values as the builtin open() function.

__call__(string)

Call self as a function.

__dict__ = mappingproxy({'__module__': 'theanolm.backend.filetypes', '__doc__': 'An object that can be passed as the "type" argument to\n    ArgumentParser.add_argument() in order to convert a path argument to a\n    file object.\n\n    If the path ends in ".gz", the file will be opened using gzip.open().\n    UTF-8 encoding will be assumed. The special path "-" means standard\n    input or output.\n\n    Keyword Arguments:\n      - mode -- A string indicating how the file is to be opened. Accepts the\n        same values as the builtin open() function.\n    ', '__init__': <function BinaryFileType.__init__>, '__call__': <function BinaryFileType.__call__>, '__repr__': <function BinaryFileType.__repr__>, '__dict__': <attribute '__dict__' of 'BinaryFileType' objects>, '__weakref__': <attribute '__weakref__' of 'BinaryFileType' objects>, '__annotations__': {}})

__init__(mode='r')

__module__ = 'theanolm.backend.filetypes'

__repr__()

Return repr(self).

__weakref__

list of weak references to the object (if defined)

class theanolm.backend.filetypes.TextFileType(mode='r')

Bases: object

An object that can be passed as the “type” argument to ArgumentParser.add_argument() in order to convert a path argument to a file object.

If the path ends in “.gz”, the file will be opened using gzip.open(). UTF-8 encoding will be assumed. The special path “-” means standard input or output.

Keyword Arguments:

• mode – A string indicating how the file is to be opened. Accepts the same values as the builtin open() function.

__call__(string)

Call self as a function.

__dict__ = mappingproxy({'__module__': 'theanolm.backend.filetypes', '__doc__': 'An object that can be passed as the "type" argument to\n    ArgumentParser.add_argument() in order to convert a path argument to a\n    file object.\n\n    If the path ends in ".gz", the file will be opened using gzip.open().\n    UTF-8 encoding will be assumed. The special path "-" means standard\n    input or output.\n\n    Keyword Arguments:\n      - mode -- A string indicating how the file is to be opened. Accepts the\n        same values as the builtin open() function.\n    ', '__init__': <function TextFileType.__init__>, '__call__': <function TextFileType.__call__>, '__repr__': <function TextFileType.__repr__>, '__dict__': <attribute '__dict__' of 'TextFileType' objects>, '__weakref__': <attribute '__weakref__' of 'TextFileType' objects>, '__annotations__': {}})

__init__(mode='r')

__module__ = 'theanolm.backend.filetypes'

__repr__()

Return repr(self).

__weakref__

list of weak references to the object (if defined)

theanolm.backend.gpu module

A module for functions related to identifying the computation devices.

theanolm.backend.gpu.get_default_device(requested)

Enumerates the devices that have been configured for Theano and returns the name of the device that will be used by default.

Devices and contexts are rather complicated in Theano. 1) If no GPU is configured, the only device that can be used is cpu and any

other requests will raise an exception.

2. If a single GPU is enabled using device=cuda0, the GPU device is None, and only None or cpu can be selected.

3. If multiple GPUs are enabled using contexts=dev0->cuda0;dev1->cuda1 and the default GPU is defined using device=cuda0, then the default device will be None and dev0 does not exist. Either None, dev1, or cpu can be selected.

4. If multiple GPUs are enabled and the default GPU is not defined, then the None device does not exist. If None is requested, the first GPU device is returned.

Parameters:

requested (str) – None to automatically selected a default device; any other selection either returns the same value or raises an exception

Return type:

str

Returns:

‘cpu’ if no GPUs are enabled, otherwise None or the name of the first GPU device

theanolm.backend.gpu.log_free_mem()

Writes the available GPU memory to the debug log.

theanolm.backend.matrixfunctions module

Functions related to creating Theano matrices.

theanolm.backend.matrixfunctions.test_value(size, high)

Creates a matrix of random numbers that can be used as a test value for a parameter to enable debugging Theano errors.

The type of high defines the type of the returned array. For integers, the range does not include the maximum value. If high is a boolean, returns an int8 array, as Theano uses int8 to represent a boolean.

Parameters:

• size (int or tuple of ints) – dimensions of the matrix

• high (int, float, or bool) – maximum value for the generated random numbers

Return type:

numpy.ndarray

Returns:

a matrix or vector containing the generated values

theanolm.backend.operations module

theanolm.backend.parameters module

A module that defines the Parameters class.

class theanolm.backend.parameters.Parameters

Bases: object

Theano Function Parameters

A dictionary of Theano shared variables. The values can be accessed through their path, which also acts as their identifier when they are saved to a HDF5 file.

__dict__ = mappingproxy({'__module__': 'theanolm.backend.parameters', '__doc__': 'Theano Function Parameters\n\n    A dictionary of Theano shared variables. The values can be accessed through\n    their path, which also acts as their identifier when they are saved to a\n    HDF5 file.\n    ', '__init__': <function Parameters.__init__>, '__getitem__': <function Parameters.__getitem__>, 'add': <function Parameters.add>, 'get_state': <function Parameters.get_state>, 'set_state': <function Parameters.set_state>, 'get_variables': <function Parameters.get_variables>, '__str__': <function Parameters.__str__>, '__dict__': <attribute '__dict__' of 'Parameters' objects>, '__weakref__': <attribute '__weakref__' of 'Parameters' objects>, '__annotations__': {}})

__getitem__(path)

Returns a shared variable given parameter path.

Parameters:

path (str) – parameter path

Return type:

SharedVariable

Returns:

the corresponding Theano shared variable

__init__()

Initializes an empty parameter dictionary.

__module__ = 'theanolm.backend.parameters'

__str__()

Returns a string representation of the parameters.

Return type:

str

Returns:

string representation of the parameters

__weakref__

list of weak references to the object (if defined)

add(path, value, device=None)

Adds a new parameter.

Parameters:

• path (str) – identifier for the shared variable in Theano and its value when stored in a HDF5 file

• value (numpy.ndarray) – initial value for the shared variable

• device (str) – if other than None, the shared variable will be kept in this device

get_state(state)

Pulls values from the shared variables into a HDF5 file.

If there already is a parameter in the file, it will be replaced, so it has to have the same number of elements.

Parameters:

state (h5py.File) – HDF5 file for storing the parameters

get_variables()

Returns a list of the shared variables.

Return type:

list of strs

Returns:

parameter paths

set_state(state)

Sets the values of the shared variables.

Requires that state contains values for all the parameters.

Parameters:

state (h5py.File) – HDF5 file that contains the parameters

theanolm.backend.probfunctions module

A module that provides functions for operating with probabilities, and defines logprob_type.

theanolm.backend.probfunctions.interpolate_linear(logprob1, logprob2, weight1)

Performs linear interpolation of two probabilities.

If a probability is so small that it will be rounded to zero, uses the decimal library. If a weight is zero, the corresponding log probability will be ignored. Otherwise if the log probability was -inf, multiplication would result in a nan.

Parameters:

• logprob1 (logprob_type) – logarithm of first input probability

• logprob2 (logprob_type) – logarithm of second input probability

• weight1 (logprob_type) – interpolation weight for the first probability

Return type:

logprob_type

Returns:

logarithm of the weighted sum of the input probabilities

theanolm.backend.probfunctions.interpolate_loglinear(logprob1, logprob2, prior1, prior2)

Performs log-linear interpolation of two probabilities.

This is not real log-linear interpolation, because we don’t normalize the result. Thus the result is not a real probability.

If a prior is zero, the corresponding log probability will be ignored. Otherwise if the log probability was -inf, multiplication would result in a nan.

Parameters:

• logprob1 (logprob_type) – first input log probability

• logprob2 (logprob_type) – second input log probability

• prior1 (logprob_type) – weight for the first log probability

• prior2 (logprob_type) – weight for the second log probability

Return type:

logprob_type

Returns:

weighted sum of the input log probabilities

Module contents