sparknlp_jsl.updateModels#

Helper class that allow us to update pretrained models located in the cache folder

Module Contents#

Classes#

UpdateModels

Helper class that allow us to update pretrained models located in the cache folder.
class UpdateModels#

Bases: object

Helper class that allow us to update pretrained models located in the cache folder.

static updateCacheModels(cache_folder='')#

Refreshes all pretrained models located in the cache pretrained folder.

Checks the existing models in the cache pretrained folder and if there is are new version for each model. If there is a new version, it will be downloaded and overwrite the existing one.

Parameters:

cache_folder (str) – Path where the models will be refreshed. i.e (“hdfs:..”,”file:…”)

static updateModels(model_names: List[str], language='en', remote_loc='clinical/models', start_date=None, end_date=None, cache_folder=None)#

Downloads all given models by filtering parameters.

If no cache_folder is specified, it will download the models in the default folder.

Parameters:

  • model_names (List[str]) – names of the models to download

  • remote_loc (str) – The remote location of the models in the s3, by default “clinical/models”.

  • language (str) – the language of the models, by default “en”

  • start_date (str) – The start date to filter the models. The start date should be in a string format that can be parsed into a valid date. Format: yyyy-MM-dd Default: None. if the value is None, it does not filter.

  • end_date (str) – The end date to filter the models. The end date should be in a string format that can be parsed into a valid date. Format: yyyy-MM-dd Default: None. if the value is None, it does not filter.

  • cache_folder (str) – Path where the models will be downloaded. i.e (“hdfs:..”,”file:…”). Default: None If no cache_folder is specified, it will download the models in the default folder.