sparknlp_jsl.transpiler.builder#

This script provides functions for preparing, building, and executing Scala and Python code. It includes functionalities for managing Spark session configuration, importing necessary libraries, and transpiling Python code to Scala.

Module Contents#

Functions#

build_python_code(py_code)

Build Python code using subprocess.

build_scala_code(scala_code)

Build Scala code using the RunTranspiledCode class.

compile_scala_code(scala_code)

Compile Scala code using the RunTranspiledCode class without execution.

prepare_python_code([spark, import_section, py_code])

Merge Python code with necessary imports and, if specified, Spark session configuration.
build_python_code(py_code)#

Build Python code using subprocess.

Parameters:

py_code (str) – Python code to be built.

Returns:

Result of building Python code.

Return type:

str

build_scala_code(scala_code)#

Build Scala code using the RunTranspiledCode class.

Parameters:

scala_code (str) – Scala code to be built.

Returns:

Result of building Scala code.

Return type:

str

compile_scala_code(scala_code)#

Compile Scala code using the RunTranspiledCode class without execution.

Parameters:

scala_code (str) – Scala code to be built.

Returns:

Result of building Scala code.

Return type:

str

prepare_python_code(spark=None, import_section='', py_code='')#

Merge Python code with necessary imports and, if specified, Spark session configuration.

Parameters:

  • spark (SparkSession or None) – If provided, it should be an instance of SparkSession.

  • import_section (bool) – Flag indicating whether import sections should be added.

  • py_code (str) – Python code to be merged.

Returns:

Merged Python code with required imports and optional Spark session configuration.

Return type:

str

Raises:

TypeError – If ‘spark’ is provided but not an instance of SparkSession.