sparknlp_jsl.pretrained#

Module Contents#

Classes#

InternalResourceDownloader

Downlod internal resources from S3

Functions#

printProgress(stop)

Prints a progress bar to the console every 2.5 seconds.
class InternalResourceDownloader#

Bases: object

Downlod internal resources from S3

static downloadModel(reader, name, language, remote_loc=None, j_dwn='InternalsPythonResourceDownloader')#

Download a model from S3.

Parameters:

  • reader (class) – The reader class to use to load the model.

  • name (str) – The name of the model to download.

  • language (str) – The language of the model to download.

  • remote_loc (str, optional) – The remote location of the model. Defaults to None.

  • j_dwn (str, optional) – The Java downloader class to use. Defaults to ‘InternalsPythonResourceDownloader’.

Returns:

The reader class with the model loaded.

Return type:

class

static downloadModelDirectly(name, remote_loc='clinical/models', unzip=True, cache_folder_path='')#

Downloads a model directly to the cache folder.

You can use to copy-paste the s3 URI from the model hub and download the model. For available s3 URI and models, please see the Models Hub.

Parameters:

  • name (str) – Name of the model or s3 URI

  • remote_loc (str, optional) – Directory of the remote Spark NLP Folder, by default “public/models”

  • unzip (Bool, optional) – Used to unzip model, by default ‘True’

  • cache_folder_path (str) –

static returnPrivateModels(annotator=None, lang=None, version=None)#

Return private models available for download.

Parameters:

  • annotator (str, optional) – The annotator to filter by. Defaults to None.

  • lang (str, optional) – The language to filter by. Defaults to None.

  • version (str, optional) – The version to filter by. Defaults to None.

Returns:

A list of private models available for download.

Return type:

list

static returnPrivatePipelines(lang=None, version=None)#

Return private pipelines available for download.

Parameters:

  • lang (str, optional) – The language to filter by. Defaults to None.

  • version (str, optional) – The version to filter by. Defaults to None.

Returns:

A list of private pipelines available for download.

Return type:

list

static showAvailableAnnotators()#

Show available annotators.

static showPrivateModels(annotator=None, lang=None, version=None)#

Show private models available for download.

Parameters:

  • annotator (str, optional) – The annotator to filter by. Defaults to None.

  • lang (str, optional) – The language to filter by. Defaults to None.

  • version (str, optional) – The version to filter by. Defaults to None.

static showPrivatePipelines(lang=None, version=None)#

Show private pipelines available for download.

Parameters:

  • lang (str, optional) – The language to filter by. Defaults to None.

  • version (str, optional) – The version to filter by. Defaults to None.

static showUnCategorizedResources()#

Show uncategorized resources available for download.

printProgress(stop)#

Prints a progress bar to the console every 2.5 seconds.

Parameters:

stop (function) – A function that returns True when the progress bar should stop.