bag
fun bagOfWords(words: Array<String>, binary: Boolean = false, columns: Array<String>? = null, tokenizer: (String) -> Array<String>): BagOfWords(source)
Bag-of-words feature transform.
Return
the BagOfWords transform.
Parameters
words
the feature words.
binary
true for presence/absence, false for word counts.
columns
the input text column names in a DataFrame (or null if applied directly on text).
tokenizer
the text tokenizer lambda.
fun bagOfWords(data: DataFrame, k: Int, vararg columns: String, tokenizer: (String) -> Array<String>): BagOfWords(source)
Learns a bag-of-words model of top-k frequent tokens in DataFrame text columns.
Return
the BagOfWords transform.
Parameters
data
training data.
k
the maximum vocabulary size.
columns
the text column names.
tokenizer
the text tokenizer lambda.