Edit on GitHub
  • John Snow Labs
  • Jun 19, 2024

State Regex Matcher

  • en
  • clinical
  • licensed
  • state
  • regexmatcher

Description

This model extracts states in clinical notes using rule-based RegexMatcherInternal annotator.

Predicted Entities

Copy S3 URI

How to use

documentAssembler = DocumentAssembler()\
    .setInputCol("text")\
    .setOutputCol("document")

sentenceDetector = SentenceDetectorDLModel.pretrained("sentence_detector_dl_healthcare","en","clinical/models")\
    .setInputCols(["document"])\
    .setOutputCol("sentence")

tokenizer = Tokenizer()\
    .setInputCols(["sentence"])\
    .setOutputCol("token")

regex_matcher = RegexMatcherInternalModel.pretrained("state_matcher","en","clinical/models") \
    .setInputCols(["sentence"])\
    .setOutputCol("state_entity")\

regex_pipeline = Pipeline().setStages([
    documentAssembler,
    sentenceDetector,
    tokenizer,
    regex_matcher])

data = spark.createDataFrame([["""California is known for its beautiful beaches and vibrant entertainment industry centered.
The Grand Canyon in Arizona is one of the most stunning natural landmarks in the world."""]]).toDF("text")

result = regex_pipeline.fit(data).transform(data)

val documentAssembler = new DocumentAssembler()
	.setInputCol("text")
	.setOutputCol("document")

val sentenceDetector = SentenceDetectorDLModel.pretrained("sentence_detector_dl_healthcare","en","clinical/models")
	.setInputCols(Array("document"))
	.setOutputCol("sentence")

val tokenizer = new Tokenizer()
	.setInputCols(Array("sentence"))
	.setOutputCol("token")

val regex_matcher = RegexMatcherInternalModel.pretrained("state_matcher","en","clinical/models")
	.setInputCols(Array("sentence"))
	.setOutputCol("state_entity")
	.setMergeOverlapping(true)

val regex_pipeline = new Pipeline().setStages(Array(
		documentAssembler,
		sentenceDetector,
		tokenizer,
		regex_matcher))

val data = Seq(""""California is known for its beautiful beaches and vibrant entertainment industry centered.
The Grand Canyon in Arizona is one of the most stunning natural landmarks in the world.""").toDF("text")

val result = regex_pipeline.fit(data).transform(data)

Results

+----------+-----+---+-----+
|     chunk|begin|end|label|
+----------+-----+---+-----+
|California|    0|  9|STATE|
|   Arizona|  124|130|STATE|
+----------+-----+---+-----+

Model Information

Model Name: state_matcher
Compatibility: Healthcare NLP 5.3.3+
License: Licensed
Edition: Official
Input Labels: [document]
Output Labels: [regex_matches]
Language: en
Size: 4.5 KB