sparknlp_jsl.transpiler.source_loader#
- The following script includes functions for processing Markdown files containing both Python and Scala code,
particularly for SparkNLP transpilation. It enables loading files, extracting Python or Scala code, updating files with transpiled Scala code, and processing Python code using a transpiler.
Module Contents#
Functions#
|
Extract Python code from a Markdown file. |
|
Extract Scala code from a Markdown file. |
|
Load two files and concatenate their contents into a dictionary. |
|
Process Python code using the transpiler. |
|
Update a Markdown file by transpiling Python code to Scala code. |
|
Update a Markdown file by replacing the Scala code. |
- get_python_code_from_md(md_file_path)#
Extract Python code from a Markdown file.
- Parameters:
md_file_path (str) – The path to the Markdown file.
- Returns:
The extracted Python code or None if not found.
- Return type:
str or None
- get_scala_code_from_md(md_file_path)#
Extract Scala code from a Markdown file.
- Parameters:
md_file_path (str) – The path to the Markdown file.
- Returns:
The extracted Scala code.
- Return type:
str
- load_file_as_dict(internal_classes, openSource_classes)#
Load two files and concatenate their contents into a dictionary.
- Parameters:
file_path1 (str) – The path to the first file.
file_path2 (str) – The path to the second file.
- Returns:
A dictionary containing the concatenated content of the two files.
- Return type:
dict
- process_python_code(python_code)#
Process Python code using the transpiler.
- Parameters:
python_code (str) – The Python code to be processed.
- Returns:
The processed Scala code.
- Return type:
str
- update_md_file(file_path)#
Update a Markdown file by transpiling Python code to Scala code.
- Parameters:
file_path (str) – The path to the Markdown file.
- Returns:
[Specify the return type and description]
- Return type:
None
- update_md_file_for_working_scala_code(md_file_path, replacement_string)#
Update a Markdown file by replacing the Scala code.
- Parameters:
md_file_path (str) – The path to the Markdown file.
replacement_string (str) – The new Scala code to replace the existing one.
- Returns:
The old Scala code or None if not found.
- Return type:
str or None