Model Utils

display_parameters

textbrewer.utils.display_parameters(model, max_level=None)[source]

Display the numbers and memory usage of module parameters.

Parameters

  • model (torch.nn.Module or dict) – the model to be inspected.

  • max_level (int or None) – The max level to display. If max_level==None, show all the levels.

Returns

A formatted string and a LayerNode object representing the model.

Data Utils

This module provides the following data augmentation methods.

masking

textbrewer.data_utils.masking(tokens, p=0.1, mask='[MASK]')[source]

Returns a new list by replacing elements in tokens by mask with probability p.

Parameters

  • tokens (list) – list of tokens or token ids.

  • p (float) – probability to mask each element in tokens.

Returns

A new list by replacing elements in tokens by mask with probability p.

deleting

textbrewer.data_utils.deleting(tokens, p=0.1)[source]

Returns a new list by deleting elements in tokens with probability p.

Parameters

  • tokens (list) – list of tokens or token ids.

  • p (float) – probability to delete each element in tokens.

Retunrns:

a new list by deleting elements in :tokens with probability p.

n_gram_sampling

textbrewer.data_utils.n_gram_sampling(tokens, p_ng=[0.2, 0.2, 0.2, 0.2, 0.2], l_ng=[1, 2, 3, 4, 5])[source]

Samples a length l from l_ng with probability distribution p_ng, then returns a random span of length l from tokens.

Parameters

  • tokens (list) – list of tokens or token ids.

  • p_ng (list) – probability distribution of the n-grams, should sum to 1.

  • l_ng (list) – specify the n-grams.

Returns

a n-gram random span from tokens.

short_disorder

textbrewer.data_utils.short_disorder(tokens, p=[0.9, 0.1, 0, 0, 0])[source]

Returns a new list by disordering tokens with probability distribution p at every possible position. Let abc be a 3-gram in tokens, there are five ways to disorder, corresponding to five probability values:

abc -> abc
abc -> bac
abc -> cba
abc -> cab
abc -> bca
Parameters

  • tokens (list) – list of tokens or token ids.

  • p (list) – probability distribution of 5 disorder types, should sum to 1.

Returns

a new disordered list

long_disorder

textbrewer.data_utils.long_disorder(tokens, p=0.1, length=20)[source]

Performs a long-range disordering. If length>1, then swaps the two halves of each span of length length in tokens; if length<=1, treats length as the relative length. For example:

>>>long_disorder([0,1,2,3,4,5,6,7,8,9,10], p=1, length=0.4)
[2, 3, 0, 1, 6, 7, 4, 5, 8, 9]
Parameters

  • tokens (list) – list of tokens or token ids.

  • p (list) – probability to swaps the two halves of a spans at possible positions.

  • length (int or float) – length of the disordered span.

Returns

a new disordered list