Skip to content

2024

  • 3 min read

Structured Output for Open Source and Local LLMS

Originally, Instructor facilitated API interactions solely via the OpenAI SDK, with an emphasis on function calling by incorporating Zod for structured data validation and serialization.

As the year progressed, we expanded our toolkit by integrating JSON mode, thus enhancing our adaptability to vision models and open source models. This advancement now enables us to support an extensive range of models, from GPT and Mistral to virtually any model accessible through Ollama and Hugging Face. For more insights into leveraging JSON mode with various models, refer back to our detailed guide on Patching.

  • 2 min read

Structured Outputs with Together and Zod

Open-source LLMS are gaining popularity, and the release of Togethers's Mistral model has made it possible to obtain structured outputs using JSON schema. Instead of relying on a model's default output mode, you can utilize JSON schema to obtain structured outputs. This approach is a time-saving alternative to extensive prompt engineering.

By the end of this blog post, you will learn how to effectively utilize instructor with Togethers. But before we proceed, let's first explore the concept of patching.

  • 2 min read

Structured Outputs with Anyscale and Zod

Open-source LLMS are gaining popularity, and the release of Anyscale's Mistral model has made it possible to obtain structured outputs using JSON schema at any scale. Instead of relying on a model's default output mode, you can utilize JSON schema to obtain structured outputs. This approach is a time-saving alternative to extensive prompt engineering.

By the end of this blog post, you will learn how to effectively utilize instructor with Anyscale. But before we proceed, let's first explore the concept of patching.