sparknlp.annotator.matcher.multi_date_matcher#

Classes#

class MultiDateMatcher[source]#

Matches standard date formats into a provided format.

Reads the following kind of dates:

"1978-01-28", "1984/04/02,1/02/1980", "2/28/79",
"The 31st of April in the year 2008", "Fri, 21 Nov 1997", "Jan 21,
‘97", "Sun", "Nov 21", "jan 1st", "next thursday", "last wednesday",
"today", "tomorrow", "yesterday", "next week", "next month",
"next year", "day after", "the day before", "0600h", "06:00 hours",
"6pm", "5:30 a.m.", "at 5", "12:59", "23:59", "1988/11/23 6pm",
"next week at 7.30", "5 am tomorrow"

For example "The 31st of April in the year 2008" will be converted into 2008/04/31.

For extended examples of usage, see the Examples.

Input Annotation types	Output Annotation type
DOCUMENT	DATE

Parameters:

dateFormat

Desired format for dates extracted, by default yyyy/MM/dd.

readMonthFirst

Whether to parse the date in mm/dd/yyyy format instead of dd/mm/yyyy, by default True.

defaultDayWhenMissing

Which day to set when it is missing from parsed input, by default 1.

anchorDateYear

Add an anchor year for the relative dates such as a day after tomorrow. If not set it will use the current year. Example: 2021

anchorDateMonth

Add an anchor month for the relative dates such as a day after tomorrow. If not set it will use the current month. Example: 1 which means January

anchorDateDay

Add an anchor day of the day for the relative dates such as a day after tomorrow. If not set it will use the current day. Example: 11

Examples

>>> import sparknlp
>>> from sparknlp.base import *
>>> from sparknlp.annotator import *
>>> from pyspark.ml import Pipeline
>>> documentAssembler = DocumentAssembler() \
...     .setInputCol("text") \
...     .setOutputCol("document")
>>> date = MultiDateMatcher() \
...     .setInputCols("document") \
...     .setOutputCol("date") \
...     .setAnchorDateYear(2020) \
...     .setAnchorDateMonth(1) \
...     .setAnchorDateDay(11) \
...     .setOutputFormat("yyyy/MM/dd")
>>> pipeline = Pipeline().setStages([
...     documentAssembler,
...     date
... ])
>>> data = spark.createDataFrame([["I saw him yesterday and he told me that he will visit us next week"]]) \
...     .toDF("text")
>>> result = pipeline.fit(data).transform(data)
>>> result.selectExpr("explode(date) as dates").show(truncate=False)
+-----------------------------------------------+
|dates                                          |
+-----------------------------------------------+
|[date, 57, 65, 2020/01/18, [sentence -> 0], []]|
|[date, 10, 18, 2020/01/10, [sentence -> 0], []]|
+-----------------------------------------------+