Detect Adverse Drug Events (langtest)

val document_assembler = new DocumentAssembler()
    .setInputCol("text")
    .setOutputCol("document")
         
val sentence_detector = new SentenceDetector()
    .setInputCols("document")
    .setOutputCol("sentence")

val tokenizer = new Tokenizer()
    .setInputCols("sentence")
    .setOutputCol("token")

val embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
    .setInputCols(Array("sentence", "token"))
    .setOutputCol("embeddings")

val ner = MedicalNerModel.pretrained("ner_ade_clinical_langtest", "en", "clinical/models")
    .setInputCols(Array("sentence", "token", "embeddings"))
    .setOutputCol("ner")

val ner_converter = new NerConverterInternal()
 	.setInputCols(Array("sentence", "token", "ner"))
 	.setOutputCol("ner_chunk")

val pipeline = new Pipeline().setStages(Array(document_assembler, 
                                              sentence_detector, 
                                              tokenizer, 
                                              embeddings_clinical, 
                                              ner, 
                                              ner_converter))


text ="""I asked my doctor what could have caused this, and he said it was probably from the Lipitor. Recently I experienced extreme stomach pain that stretched to my side and back. The stomach pain came suddenly and it would come and go for a week before it became so bad I could not sleep. I had just gotten over a urinary tract infection (one of the side-effects) and I thought it had returned, but the symptoms were different. I still had the urge to urinate constantly, but it would come and go, and I would have no symptoms for a day, then it would return again. I remembered reading the pamphlet that comes with the prescription drugs, and it mentioning some of the symptoms that I was experiencing. I stopped taking the Lipitor for a day, and I did not have any more stomach pain or urgency. I also had been experiencing lack of energy for quite some time, and I attributed this to stress, but after reading this website, I feel it was due to the Lipitor also. I don't think I will take this drug anymore, and since I read that taking vitamin C can help you with your cholesterol, I think I will try this method instead. I think there should be a better alternative to lowering cholesterol than such a potent drug that can cause so many side effects. I don't want to be a case-study when they finally take this drug off the market."""

val data = Seq(text).toDS.toDF("text")

val result = pipeline.fit(data).transform(data)