Packages

package bpe

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. class RobertaTokenizer extends BpeTokenizer
  2. case class SpecialToken(content: String, id: Int, singleWord: Boolean = false, lstrip: Boolean = false, rstrip: Boolean = false) extends Product with Serializable

Value Members

  1. object BpeTokenizer

Ungrouped