Description
This pipeline can be used to deidentify PHI information from medical texts. The PHI information will be masked and obfuscated in the resulting text. The pipeline can mask and obfuscate AGE, CONTACT, DATE, ID, LOCATION, NAME, PROFESSION, CITY, COUNTRY, DOCTOR, HOSPITAL, IDNUM, MEDICALRECORD, ORGANIZATION, PATIENT, PHONE, PROFESSION, STREET, USERNAME, ZIP, ACCOUNT, LICENSE, VIN, SSN, DLN, PLATE, IPADDR entities.
Predicted Entities
ACCOUNT, AGE, BIOID, CITY, CONTACT, COUNTRY, DATE, DEVICE, DLN, DOCTOR, EMAIL, FAX, HEALTHPLAN, HOSPITAL, ID, IDNUM, LICENSE, LOCATION, LOCATION-OTHER, MEDICALRECORD, NAME, ORGANIZATION, PATIENT, PHONE, PLATE, PROFESSION, SSN, STATE, STREET, URL, USERNAME, VIN, ZIP
Live Demo Open in Colab Copy S3 URI
How to use
from sparknlp.pretrained import PretrainedPipeline
deid_pipeline = PretrainedPipeline("clinical_deidentification", "en", "clinical/models")
deid_pipeline.annotate("Record date : 2093-01-13, David Hale, M.D. IP: 203.120.223.13. The driver's license no:A334455B. the SSN:324598674 and e-mail: hale@gmail.com. Name : Hendrickson, Ora MR. # 719435 Date : 01/13/93. PCP : Oliveira, 25 years-old. Record date : 2079-11-09, Patient's VIN : 1HGBH41JXMN109286.")
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline
val deid_pipeline = PretrainedPipeline("clinical_deidentification","en","clinical/models")
val result = pipeline.annotate("Record date : 2093-01-13, David Hale, M.D. IP: 203.120.223.13. The driver's license no:A334455B. the SSN:324598674 and e-mail: hale@gmail.com. Name : Hendrickson, Ora MR. # 719435 Date : 01/13/93. PCP : Oliveira, 25 years-old. Record date : 2079-11-09, Patient's VIN : 1HGBH41JXMN109286.")
import nlu
nlu.load("en.de_identify.clinical_pipeline").predict("""Record date : 2093-01-13, David Hale, M.D. IP: 203.120.223.13. The driver's license no:A334455B. the SSN:324598674 and e-mail: hale@gmail.com. Name : Hendrickson, Ora MR. # 719435 Date : 01/13/93. PCP : Oliveira, 25 years-old. Record date : 2079-11-09, Patient's VIN : 1HGBH41JXMN109286.""")
Results
{'sentence': ['Record date : 2093-01-13, David Hale, M.D.',
'IP: 203.120.223.13.',
'The driver's license no:A334455B.',
'the SSN:324598674 and e-mail: hale@gmail.com.',
'Name : Hendrickson, Ora MR. # 719435 Date : 01/13/93.',
'PCP : Oliveira, 25 years-old.',
'Record date : 2079-11-09, Patient's VIN : 1HGBH41JXMN109286.'],
'masked': ['Record date : <DATE>, <NAME>, M.D.',
'IP: <IPADDR>.',
'The driver's license <DLN>.',
'the <SSN> and e-mail: <EMAIL>.',
'Name : <NAME> MR. # <ID> Date : <DATE>.',
'PCP : <NAME>, <AGE> years-old.',
'Record date : <DATE>, Patient's VIN : <VIN>.'],
'obfuscated': ['Record date : 2093-02-13, Shella Solan, M.D.',
'IP: 333.333.333.333.',
'The driver's license O497302436569.',
'the SSN-539-29-1060 and e-mail: Keith@google.com.',
'Name : Cindy Nakai MR. # I7396944 Date : 06-11-1985.',
'PCP : Benigno Paganini, 3 years-old.',
'Record date : 2079-12-30, Patient's VIN : 5eeee44ffff555666.'],
'ner_chunk': ['2093-01-13',
'David Hale',
'no:A334455B',
'SSN:324598674',
'Hendrickson, Ora',
'719435',
'01/13/93',
'Oliveira',
'25',
'2079-11-09',
'1HGBH41JXMN109286']}
Model Information
| Model Name: | clinical_deidentification |
| Type: | pipeline |
| Compatibility: | Healthcare NLP 3.0.3+ |
| License: | Licensed |
| Edition: | Official |
| Language: | en |
Included Models
- DocumentAssembler
- TokenizerModel
- LemmatizerModel
- Finisher