Packages

package qa

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. trait GPTGenerationParams extends Params
  2. class MedicalQuestionAnswering extends MedicalTextGenerator

    MedicalQuestionAnswering is a GPT based model for answering questions given a context.

    MedicalQuestionAnswering is a GPT based model for answering questions given a context. Unlike span based models, it generates the answers to the questions, rather than selecting phrases from the given context. The model is capable of answering various types of questions, including yes-no or full text ones.

    Pretrained models can be loaded with pretrained of the companion object:

    val medQA = MedicalQuestionAnswering.pretrained()
  .setInputCols("question_document", "context_document")
  .setOutputCol("answer")

    For available pretrained models please see the Models Hub.

    Example

     import spark.implicits._

 val documentAssembler = new MultiDocumentAssembler()
   .setInputCols("question", "context")
   .setOutputCols("document_question", "document_context")

val medQA = MedicalQuestionAnswering.pretrained()
  .setInputCols("question_document", "context_document")
  .setOutputCol("answer")
  .setQuestionType("short")

 val pipeline = new Pipeline()
   .setStages(Array(
     documentAssembler,
     medQA))

 val model = pipeline.fit(Seq("", "").toDS.toDF("question", "context"))
 val results = model.transform(
   Seq(
   ("Should chest wall irradiation be included after mastectomy and negative node breast cancer?",
   """
       |This study aims to evaluate local failure patterns in node negative breast cancer patients treated with
       |post-mastectomy radiotherapy including internal mammary chain only. Retrospective analysis of 92 internal
       |or central-breast node-negative tumours with mastectomy and external irradiation of the internal mammary
       |chain at the dose of 50 Gy, from 1994 to 1998. Local recurrence rate was 5 % (five cases). Recurrence
       |sites were the operative scare and chest wall. Factors associated with increased risk of local failure
       |were age<or = 40 years and tumour size greater than 20mm, without statistical significance.
       |""".stripMargin)
   ).toDS.toDF("question", "context"))

 results
   .selectExpr("answer.result")
   .show(truncate = false)

 +-------+
 |result |
 +-------+
 |[yes]  |
 +-------+
    See also

    https://academic.oup.com/bib/article/23/6/bbac409/6713511 for details about using medical text generation with GPT

  3. class NerQuestionGenerator extends AnnotatorModel[NerQuestionGenerator] with HasSimpleAnnotate[NerQuestionGenerator] with CheckLicense

    Automatically generates questions for NER.

    Automatically generates questions for NER.

    This annotator takes a NER chunk (obtained by, e.g., NerConverterInternal) and generates a questions based on two entity types, a pronoun and a strategy.

    The question is generated in the form of [QUESTIONPRONOUN] [ENTITY1] [ENTITY2] [QUESTIONMARK].

    The generated question can be used by QuestionAnswerer annotator to find answers to the generated questions.

    Pretrained models can be loaded with pretrained of the companion object:

    val medQA = NerQuestionGenerator.pretrained()
  .setInputCols("ner_chunk")
  .setOutputCol("question")
  4. trait ReadMedicalQuestionAnsweringModel extends ReadTensorflowModel with ReadSentencePieceModel with InternalReadOnnxModel
  5. trait ReadablePretrainedMedicalQuestionAnsweringModel extends ParamsAndFeaturesReadable[MedicalQuestionAnswering] with HasPretrained[MedicalQuestionAnswering]

Value Members

  1. object MedicalQuestionAnswering extends ReadablePretrainedMedicalQuestionAnsweringModel with ReadMedicalQuestionAnsweringModel with Serializable

Ungrouped