Get Tokenizer for tokenizing train/test data

Source: R/keras_helpers.R

Get Tokenizer for tokenizing train/test data

get_tokenizer(data, max_words)

Arguments

data

Data to fit tokenizer on.
max_words

Maximum number of words to consider using word frequency measure.

Value

tokenizer object