Deploy LLMs as Private Databricks Endpoints

The LLMs listed below are available on Databricks Marketplace as Model Serving Endpoints.

Medical Visual LLM - 8B

Medical-LLM-14B

Medical LLM - Small

Medical-Visual-LLM-30B

Medical-Reasoning-LLM-32B

Medical-LLM-Medium

Deployment Instructions

1. Click Get access on the model listing page in Databricks Marketplace.

2. Once access is granted, click the Open button to view the model and available versions.

3. Select the model version with the production alias (recommended for stable deployments) and click Serve this model.

4. In the Create serving endpoint configuration:

   • Compute type: Select the appropriate compute type based on your workload requirements. Refer to the memory requirements table to determine the approximate memory needed for your selected model.

   • Environment variables: Add the following required environment variable:

     SPARK_NLP_LICENSE="your-license-key"
     

5. Complete the serving endpoint configuration and click Create.

6. Wait for the endpoint to become active.

Note: Each model listing includes a detailed notebook with comprehensive instructions for creating serving endpoints using the Databricks API, along with inference examples and supported formats.