c

com.johnsnowlabs.ml.tensorflow

TensorflowMarian

class TensorflowMarian extends Serializable

MarianTransformer: Fast Neural Machine Translation

MarianTransformer uses models trained by MarianNMT.

Marian is an efficient, free Neural Machine Translation framework written in pure C++ with minimal dependencies. It is mainly being developed by the Microsoft Translator team. Many academic (most notably the University of Edinburgh and in the past the Adam Mickiewicz University in Poznań) and commercial contributors help with its development.

It is currently the engine behind the Microsoft Translator Neural Machine Translation services and being deployed by many companies, organizations and research projects (see below for an incomplete list).

Sources : MarianNMT https://marian-nmt.github.io/ Marian: Fast Neural Machine Translation in C++ https://www.aclweb.org/anthology/P18-4020/

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TensorflowMarian
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TensorflowMarian(tensorflow: TensorflowWrapper, sppSrc: SentencePieceWrapper, sppTrg: SentencePieceWrapper, configProtoBytes: Option[Array[Byte]] = None, signatures: Option[Map[String, String]] = None)

    tensorflow

    LanguageDetectorDL Model wrapper with TensorFlow Wrapper

    sppSrc

    Contains the vocabulary for the target language.

    sppTrg

    Contains the vocabulary for the source language

    configProtoBytes

    Configuration for TensorFlow session

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val _tfMarianSignatures: Map[String, String]
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  7. def decode(sentences: Array[Array[Int]], vocabsArray: Array[String]): Seq[String]
  8. def encode(sentences: Seq[Annotation], normalizer: MosesPunctNormalizer, maxSeqLength: Int, vocabsArray: Array[String], langId: Int, unknownTokenId: Int, eosTokenId: Int): Seq[Array[Int]]
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. def generateSeq2Seq(sentences: Seq[Annotation], batchSize: Int = 10, maxInputLength: Int, maxOutputLength: Int, vocabs: Array[String], langId: String): Array[Annotation]

    generate seq2seq via encoding, generating, and decoding

    generate seq2seq via encoding, generating, and decoding

    sentences

    none empty Annotation

    batchSize

    size of baches to be process at the same time

    maxInputLength

    maximum length for input

    maxOutputLength

    maximum length for output

    vocabs

    list of all vocabs

    langId

    language id for multi-lingual models

  13. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. def process(batch: Seq[Array[Int]], maxOutputLength: Int, paddingTokenId: Int, eosTokenId: Int, vocabSize: Int): Array[Array[Int]]
  20. val sppSrc: SentencePieceWrapper
  21. val sppTrg: SentencePieceWrapper
  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. val tensorflow: TensorflowWrapper
  24. def toString(): String
    Definition Classes
    AnyRef → Any
  25. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped