API - Load, Save Model and Data

Load benchmark dataset, save and restore model, save and load variables. TensorFlow provides .ckpt file format to save and restore the models, while we suggest to use standard python file format .npz to save models for the sake of cross-platform.

# save model as .ckpt
saver = tf.train.Saver()
save_path = saver.save(sess, "model.ckpt")
# restore model from .ckpt
saver = tf.train.Saver()
saver.restore(sess, "model.ckpt")

# save model as .npz
tl.files.save_npz(network.all_params , name='model.npz')

# restore model from .npz
load_params = tl.files.load_npz(path='', name='model.npz')
tl.files.assign_params(sess, load_params, network)

# you can assign the pre-trained parameters as follow
# 1st parameter
tl.files.assign_params(sess, [load_params[0]], network)
# the first three parameters
tl.files.assign_params(sess, load_params[:3], network)
load_mnist_dataset([shape, path]) Automatically download MNIST dataset and return the training, validation and test set with 50000, 10000 and 10000 digit images respectively.
load_cifar10_dataset([shape, path, …]) The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class.
load_ptb_dataset([path]) Penn TreeBank (PTB) dataset is used in many LANGUAGE MODELING papers, including “Empirical Evaluation and Combination of Advanced Language Modeling Techniques”, “Recurrent Neural Network Regularization”.
load_matt_mahoney_text8_dataset([path]) Download a text file from Matt Mahoney’s website if not present, and make sure it’s the right size.
load_imbd_dataset
load_nietzsche_dataset([path]) Load Nietzsche dataset.
load_wmt_en_fr_dataset([path]) It will download English-to-French translation data from the WMT‘15 Website (10^9-French-English corpus), and the 2013 news test from the same site as development set.
save_npz([save_list, name, sess]) Input parameters and the file name, save parameters into .npz file.
load_npz([path, name]) Load the parameters of a Model saved by tl.files.save_npz().
assign_params(sess, params, network) Assign the given parameters to the TensorLayer network.
load_and_assign_npz([sess, name, network]) Load model from npz and assign to a network.
save_any_to_npy([save_dict, name]) Save variables to .npy file.
load_npy_to_any([path, name]) Load .npy file.
npz_to_W_pdf([path, regx]) Convert the first weight matrix of .npz file to .pdf by using tl.visualize.W().
load_file_list([path, regx, printable]) Return a file list in a folder by given a path and regular expression.
load_folder_list([path]) Return a folder list in a folder by given a folder path.
exists_or_mkdir(path[, verbose]) Check a folder by given name, if not exist, create the folder and return False, if directory exists, return True.
maybe_download_and_extract(filename, …[, …]) Checks if file exists in working_directory otherwise tries to dowload the file, and optionally also tries to extract the file if format is “.zip” or “.tar”

Load dataset functions

MNIST

tensorlayer.files.load_mnist_dataset(shape=(-1, 784), path='data/mnist/')[source]

Automatically download MNIST dataset and return the training, validation and test set with 50000, 10000 and 10000 digit images respectively.

Parameters:
shape : tuple

The shape of digit images, defaults to (-1,784)

path : string

Path to download data to, defaults to data/mnist/

Examples

>>> X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1,784))
>>> X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 28, 28, 1))

CIFAR-10

tensorlayer.files.load_cifar10_dataset(shape=(-1, 32, 32, 3), path='data/cifar10/', plotable=False, second=3)[source]

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

Parameters:
shape : tupe

The shape of digit images: e.g. (-1, 3, 32, 32) , (-1, 32, 32, 3) , (-1, 32*32*3)

plotable : True, False

Whether to plot some image examples.

second : int

If plotable is True, second is the display time.

path : string

Path to download data to, defaults to data/cifar10/

Notes

CIFAR-10 images can only be display without color change under uint8. >>> X_train = np.asarray(X_train, dtype=np.uint8) >>> plt.ion() >>> fig = plt.figure(1232) >>> count = 1 >>> for row in range(10): >>> for col in range(10): >>> a = fig.add_subplot(10, 10, count) >>> plt.imshow(X_train[count-1], interpolation=’nearest’) >>> plt.gca().xaxis.set_major_locator(plt.NullLocator()) # 不显示刻度(tick) >>> plt.gca().yaxis.set_major_locator(plt.NullLocator()) >>> count = count + 1 >>> plt.draw() >>> plt.pause(3)

References

Examples

>>> X_train, y_train, X_test, y_test = tl.files.load_cifar10_dataset(shape=(-1, 32, 32, 3), plotable=True)

Penn TreeBank (PTB)

tensorlayer.files.load_ptb_dataset(path='data/ptb/')[source]

Penn TreeBank (PTB) dataset is used in many LANGUAGE MODELING papers, including “Empirical Evaluation and Combination of Advanced Language Modeling Techniques”, “Recurrent Neural Network Regularization”.

It consists of 929k training words, 73k validation words, and 82k test words. It has 10k words in its vocabulary.

In “Recurrent Neural Network Regularization”, they trained regularized LSTMs of two sizes; these are denoted the medium LSTM and large LSTM. Both LSTMs have two layers and are unrolled for 35 steps. They initialize the hidden states to zero. They then use the final hidden states of the current minibatch as the initial hidden state of the subsequent minibatch (successive minibatches sequentially traverse the training set). The size of each minibatch is 20.

The medium LSTM has 650 units per layer and its parameters are initialized uniformly in [−0.05, 0.05]. They apply 50% dropout on the non-recurrent connections. They train the LSTM for 39 epochs with a learning rate of 1, and after 6 epochs they decrease it by a factor of 1.2 after each epoch. They clip the norm of the gradients (normalized by minibatch size) at 5.

The large LSTM has 1500 units per layer and its parameters are initialized uniformly in [−0.04, 0.04]. We apply 65% dropout on the non-recurrent connections. They train the model for 55 epochs with a learning rate of 1; after 14 epochs they start to reduce the learning rate by a factor of 1.15 after each epoch. They clip the norm of the gradients (normalized by minibatch size) at 10.

Parameters:
path : : string

Path to download data to, defaults to data/ptb/

Returns:
train_data, valid_data, test_data, vocabulary size

Examples

>>> train_data, valid_data, test_data, vocab_size = tl.files.load_ptb_dataset()

Matt Mahoney’s text8

tensorlayer.files.load_matt_mahoney_text8_dataset(path='data/mm_test8/')[source]

Download a text file from Matt Mahoney’s website if not present, and make sure it’s the right size. Extract the first file enclosed in a zip file as a list of words. This dataset can be used for Word Embedding.

Parameters:
path : : string

Path to download data to, defaults to data/mm_test8/

Returns:
word_list : a list

a list of string (word).

e.g. […. ‘their’, ‘families’, ‘who’, ‘were’, ‘expelled’, ‘from’, ‘jerusalem’, …]

Examples

>>> words = tl.files.load_matt_mahoney_text8_dataset()
>>> print('Data size', len(words))

IMBD

Nietzsche

tensorlayer.files.load_nietzsche_dataset(path='data/nietzsche/')[source]

Load Nietzsche dataset. Returns a string.

Parameters:
path : string

Path to download data to, defaults to data/nietzsche/

Examples

>>> see tutorial_generate_text.py
>>> words = tl.files.load_nietzsche_dataset()
>>> words = basic_clean_str(words)
>>> words = words.split()

English-to-French translation data from the WMT‘15 Website

tensorlayer.files.load_wmt_en_fr_dataset(path='data/wmt_en_fr/')[source]

It will download English-to-French translation data from the WMT‘15 Website (10^9-French-English corpus), and the 2013 news test from the same site as development set. Returns the directories of training data and test data.

Parameters:
path : string

Path to download data to, defaults to data/wmt_en_fr/

Notes

Usually, it will take a long time to download this dataset.

References

  • Code modified from /tensorflow/models/rnn/translation/data_utils.py

Load and save network

Save network as .npz

tensorlayer.files.save_npz(save_list=[], name='model.npz', sess=None)[source]

Input parameters and the file name, save parameters into .npz file. Use tl.utils.load_npz() to restore.

Parameters:
save_list : a list

Parameters want to be saved.

name : a string or None

The name of the .npz file.

sess : None or Session

Notes

If you got session issues, you can change the value.eval() to value.eval(session=sess)

References

Examples

>>> tl.files.save_npz(network.all_params, name='model_test.npz', sess=sess)
... File saved to: model_test.npz
>>> load_params = tl.files.load_npz(name='model_test.npz')
... Loading param0, (784, 800)
... Loading param1, (800,)
... Loading param2, (800, 800)
... Loading param3, (800,)
... Loading param4, (800, 10)
... Loading param5, (10,)
>>> put parameters into a TensorLayer network, please see assign_params()

Load network from .npz

tensorlayer.files.load_npz(path='', name='model.npz')[source]

Load the parameters of a Model saved by tl.files.save_npz().

Parameters:
path : a string

Folder path to .npz file.

name : a string or None

The name of the .npz file.

Returns:
params : list

A list of parameters in order.

References

Examples

  • See save_npz and assign_params

Assign parameters to network

tensorlayer.files.assign_params(sess, params, network)[source]

Assign the given parameters to the TensorLayer network.

Parameters:
sess : TensorFlow Session
params : a list

A list of parameters in order.

network : a Layer class

The network to be assigned

References

Examples

>>> Save your network as follow:
>>> tl.files.save_npz(network.all_params, name='model_test.npz')
>>> network.print_params()
...
... Next time, load and assign your network as follow:
>>> sess.run(tf.initialize_all_variables()) # re-initialize, then save and assign
>>> load_params = tl.files.load_npz(name='model_test.npz')
>>> tl.files.assign_params(sess, load_params, network)
>>> network.print_params()

Load and assign parameters to network

tensorlayer.files.load_and_assign_npz(sess=None, name=None, network=None)[source]

Load model from npz and assign to a network.

Parameters:
sess : TensorFlow Session
name : string

Model path.

network : a Layer class

The network to be assigned

Returns:
Returns False if faild to model is not exist.

Examples

>>> tl.files.load_and_assign_npz(sess=sess, name='net.npz', network=net)

Load and save variables

Save variables as .npy

tensorlayer.files.save_any_to_npy(save_dict={}, name='file.npy')[source]

Save variables to .npy file.

Examples

>>> tl.files.save_any_to_npy(save_dict={'data': ['a','b']}, name='test.npy')
>>> data = tl.files.load_npy_to_any(name='test.npy')
>>> print(data)
... {'data': ['a','b']}

Load variables from .npy

tensorlayer.files.load_npy_to_any(path='', name='file.npy')[source]

Load .npy file.

Examples

  • see save_any_to_npy()

Visualizing npz file

tensorlayer.files.npz_to_W_pdf(path=None, regx='w1pre_[0-9]+\\.(npz)')[source]

Convert the first weight matrix of .npz file to .pdf by using tl.visualize.W().

Parameters:
path : a string or None

A folder path to npz files.

regx : a string

Regx for the file name.

Examples

>>> Convert the first weight matrix of w1_pre...npz file to w1_pre...pdf.
>>> tl.files.npz_to_W_pdf(path='/Users/.../npz_file/', regx='w1pre_[0-9]+\.(npz)')

Folder functions

Load file list from folder

tensorlayer.files.load_file_list(path=None, regx='\\.npz', printable=True)[source]

Return a file list in a folder by given a path and regular expression.

Parameters:
path : a string or None

A folder path.

regx : a string

The regx of file name.

printable : boolean, whether to print the files infomation.

Examples

>>> file_list = tl.files.load_file_list(path=None, regx='w1pre_[0-9]+\.(npz)')

Load folder list from folder

tensorlayer.files.load_folder_list(path='')[source]

Return a folder list in a folder by given a folder path.

Parameters:
path : a string or None

A folder path.

Check and Create folder

tensorlayer.files.exists_or_mkdir(path, verbose=True)[source]

Check a folder by given name, if not exist, create the folder and return False, if directory exists, return True.

Parameters:
path : a string

A folder path.

verbose : boolean

If True, prints results, deaults is True

Returns:
True if folder exist, otherwise, returns False and create the folder

Examples

>>> tl.files.exists_or_mkdir("checkpoints/train")

Download or extract

tensorlayer.files.maybe_download_and_extract(filename, working_directory, url_source, extract=False, expected_bytes=None)[source]

Checks if file exists in working_directory otherwise tries to dowload the file, and optionally also tries to extract the file if format is “.zip” or “.tar”

Parameters:
filename : string

The name of the (to be) dowloaded file.

working_directory : string

A folder path to search for the file in and dowload the file to

url : string

The URL to download the file from

extract : bool, defaults to False

If True, tries to uncompress the dowloaded file is “.tar.gz/.tar.bz2” or “.zip” file

expected_bytes : int/None

If set tries to verify that the downloaded file is of the specified size, otherwise raises an Exception, defaults to None which corresponds to no check being performed

Returns
———-
filepath to dowloaded (uncompressed) file

Examples

>>> down_file = tl.files.maybe_download_and_extract(filename = 'train-images-idx3-ubyte.gz',
                                                    working_directory = 'data/',
                                                    url_source = 'http://yann.lecun.com/exdb/mnist/')
>>> tl.files.maybe_download_and_extract(filename = 'ADEChallengeData2016.zip',
                                        working_directory = 'data/',
                                        url_source = 'http://sceneparsing.csail.mit.edu/data/',
                                        extract=True)