Spark OCR release notes

7.0.0

Release date: 07-10-2026

Visual NLP 7.0.0 Release Notes 🕶️

We are glad to announce that Visual NLP 7.0.0 has been released! This is a release with multiple small, but impactful changes. 📢📢📢

Main Changes 🔴

• Added support for latest Spark 4.X versions, and continue with our efforts to support Scala 2.13 and Java 17.
• Dicom Processing improvements, and also improvements in our Pathology (SVS) Processing capabilities.
• New models and pipelines.
• A new Infrastructure Stack (Cloudformation) to run SVS and Dicom de-identification in AWS, and new instructions on how to run on Databricks.

Important changes in start() function

Following the guidelines of open source Spark-NLP, JAR artifact selection in start() function is now determined by PySpark version, as follows,

Dicom Changes

• Updates the DicomPretrainedPipeline to support strategy files used in de-identification of Dicom tags. Also, DicomPretrainedPipeline now supports load()/save().
• New replaceWithMapping action in DicomMetadataDeidentifier: replaces tag values from an external mapping column (externalMapping), with optional nested-tag handling.
• Overlay extraction off by default (new extractOverlay param).
• Fixes for YBR 8-bit compression, compression ratio calculation, JPEG baseline 12-bit and Monochrome1 white pixels.
• DicomDrawRegions has been fixed for reliability and the dependency on python-gdcm has been removed, making setup of the Visual NLP Python package easier.

Pathology (SVS) files improvement

• Added support for non-square tile grids that enables for processing of a wider variety of files.
• Improved heuristics in layer selection for image deid to reduce processing times and memory peaks.
• Supports in-place masking of large SVS files for improved memory management.
• remove_phi() now fails on files with no readable pages to help spot broken input files.

Databricks Cluster Setup Instructions

Updated instructions for Databricks cluster setup according to latest Databricks releases. Check the documentation here, and related notebook VisualNLP_DB_Cluster_Setup.ipynb.

New Infrastructure Stack for SVS/DICOM

In a nutshell, this architecture works as follows,

S3 → EventBridge → Lambda → AWS Batch (EC2, c7a.2xlarge) → container → S3

• S3: stores the data.
• EventBridge: notices that something happened.
• Lambda: decides what should be processed and submits a Batch job.
• AWS Batch: allocates compute and starts your container.
• A container: reads from S3, performs inference, and writes results back to S3.

The DICOM version follows the same infrastructure pattern as the SVS one, targeting DICOM files instead of .svs whole-slide images.

Links for both SVS and DICOM versions can be found here.

New Layout architecture & model

New ImageObjectDetectorRtDetr architecture: RT-DETR / RT-DETRv2 ONNX object detector with COCO labels by default, GPU option and LightPipeline support. It can detect 17 layout classes:

caption, footnote, formula, list_item, page_footer, page_header, picture, section_header, table, text, title, document_index, code, checkbox_selected, checkbox_unselected, form, key_value_region.

To see it in action check SparkOcrLayoutDetectionRtDetr.ipynb.

New PDF Deidentification Pipeline

We’re releasing a new Handwriting-Aware PDF deidentification pipeline, pdf_deid_multi_model_handwritten_context_pipeline. This pipeline is an enhancement over our previous pdf_deid_multi_model_context_pipeline, which powers our PDF-deid AWS listing as well.
To see this pipeline in action, please check out the SparkOcrPdfDeIdentificationPipelines.ipynb notebook.

Dicom Deidentification Skill

Agentic Pipeline building for Dicom Deidentification is here! The Visual NLP DICOM Skill gives an LLM a grounded way to generate John Snow Labs Visual NLP pipeline examples for specific DICOM de-identification tasks. Check blogpost here.

New Pipelines

Added form support through doc_data_loader_digital_hybrid_tables_forms_vlm1 and doc_data_loader_digital_hybrid_tables_forms_vlm2, and the full list of data loader pipelines is:

You can see these pipelines in action here: SparkOcrDocDataLoaderPipelines.ipynb.

Compatibility:

Spark-NLP 7.0.0, and Spark-NLP for Healthcare 7.0.0.

Previous versions