Substrate

Substrate is designed for running and composing multi-step AI programs. It provides a unified modular API that includes optimized models, file and vector storage, a code interpreter, and logical control flow to ensure parallelism, throughput, and data locality.

Substrate lets you describe complex AI programs in a natural, imperative style without having to write a lot of code. This approach ensures perfect parallelism, opportunistic batching, zero roundtrips, and near-instant communication between nodes, making it particularly well-suited for high-performance AI workloads.

Some of the benefits of Substrate include:

• Connect nodes: Describe graphs in a natural imperative style, automatically connecting nodes and running high-performance programs.
• Write less code: Run more inference and do less infrastructure work.
• Microsecond communication: Run entire graphs of many nodes on a single machine, with very fast communication between tasks.

Substrate offers a variety of pricing tiers to suit different needs:

• Starter: Free with $8 starter credits, then $30/month + usage, with an 8 node concurrency limit and access to the community Slack.
• Pro: $500/month + usage, with a 64 node concurrency limit and priority input to the roadmap.
• Enterprise: Contact sales for a custom plan with maximum node concurrency, custom models, dedicated instances, and support through private Slack.
• Self-hosted: Contact sales for a self-hosted option that runs Substrate on your cluster, allowing custom model deployment and generated SDKs.

Pricing for usage varies by model type:

• Language Models: $0.50 per 1M tokens for Mistral 7B, Llama 3 8B, and FireLLaVA 13B, and $1.00 per 1M tokens for Mixtral 8x7B and $1.50 per 1M tokens for Llama 3 70B.
• Image Models: $0.02 per image for all models.
• Segmentation Models: $0.02 per image for all models.
• Audio Models: $0.01 per minute for TranscribeSpeech and $5 per 1M characters for GenerateSpeech.
• Embedding Models: $0.02 per 1M tokens for Jina v2 and $0.01 per 1M tokens for CLIP.
• Vector Stores: $0.10 per GB vector storage per day.
• Code Interpreter: $0.03 per session for CPU and custom pricing for GPU.

For more information on how to get started with their AI program management solutions, check out the Substrate website.

Published on June 21, 2024