Description
Google’s T5 small fine-tuned on WikiSQL for English to SQL translation. Will generate SQL code from natural language input when task is set it to “translate English to SQL:”.
Predicted Entities
Live Demo Open in Colab Download Copy S3 URI
How to use
import sparknlp
from sparknlp.base import *
from sparknlp.annotator import *
spark = sparknlp.start()
documentAssembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")
t5 = T5Transformer.pretrained("t5_small_wikiSQL") \
.setTask("translate English to SQL:") \
.setInputCols(["documents"]) \
.setMaxOutputLength(200) \
.setOutputCol("sql")
pipeline = Pipeline().setStages([documentAssembler, t5])
data = spark.createDataFrame([["How many customers have ordered more than 2 items?"]]).toDF("text")
result = pipeline.fit(data).transform(data)
result.select("sql.result").show(truncate=False)
import spark.implicits._
import com.johnsnowlabs.nlp.base.DocumentAssembler
import com.johnsnowlabs.nlp.annotators.seq2seq.T5Transformer
import org.apache.spark.ml.Pipeline
val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("documents")
val t5 = T5Transformer.pretrained("t5_small_wikiSQL")
.setInputCols("documents")
.setMaxOutputLength(200)
.setOutputCol("sql")
val pipeline = new Pipeline().setStages(Array(documentAssembler, t5))
val data = Seq("How many customers have ordered more than 2 items?")
.toDF("text")
val result = pipeline.fit(data).transform(data)
result.select("sql.result").show(false)
import nlu
nlu.load("en.t5.wikiSQL").predict("""How many customers have ordered more than 2 items?""")
Results
+----------------------------------------------------+
|result |
+----------------------------------------------------+
|[SELECT COUNT Customers FROM table WHERE Orders > 2]|
+----------------------------------------------------+
Model Information
| Model Name: | t5_small_wikiSQL |
| Compatibility: | Spark NLP 3.4.0+ |
| License: | Open Source |
| Edition: | Official |
| Input Labels: | [documents] |
| Output Labels: | [sql] |
| Language: | en |
| Size: | 262.1 MB |
Data Source
Model originally from the transformer model of Manuel Romero/mrm8488. https://huggingface.co/mrm8488/t5-small-finetuned-wikiSQL