Frequently Asked Questions

Here you can find answers to common questions about Codex.

Can’t find the answer? Join our Slack community for live support or contact us at support@cleanlab.ai.

This page is for Developers and Managers. If your SMEs are having trouble using Codex’s Web Interface, contact us for a training session (only takes 30min).

How does Codex compare to an LLM Evals platform?

Evals platforms help you catch bad AI responses and understand the performance of your AI application, but do not help you directly improve the application.

Codex is a platform to directly improve AI responses.

In addition, detecting bad AI responses in Evals platforms often requires manual work and writing evaluation functions. Detecting bad/unhelpful AI responses in Codex can be done automatically (see Codex’s as-a-Backup integration option). Codex also provides the interfaces to remediate these bad AI responses, ensuring they never occur again in your AI application.

Why not have SMEs directly edit the RAG Knowledge Base or documents?

There are several reasons you might prefer to use Codex over attempting to have SMEs at your company directly edit the Knowledge Base or documents contained therein:

Guaranteed remediation. When a bad AI response is discovered, it is not guaranteed that editing a document will prevent this bad response from occuring again. Bad AI responses are often the byproduct of imperfect search/retrieval or LLM hallucination. Simply having the right data in your Knowledge Base may not address these root causes. The specific formatting of documents/chunks is critical and SMEs may not know the right formatting for their edits to best prevent bad AI responses like this.
Ease/Speed. It is much easier for SMEs to add Answers to specific user queries into the Codex interface, compared to updating documents to ensure the answer is properly reflected. The former just requires typing in the answer; the latter requires: finding the right documents, brainstorming how to best include new information without making things too dense and preserving document formatting.
Instant remediation enables testing/refinement. Directly editing documents / data sources for the RAG Knowledge Base typically does not immediately improve the RAG system. It can take time for such edits to propagate into the AI application, which makes it hard for SMEs to test whether their improvements are effective. With Codex, answers provided by SMEs are instantly available to the AI, which makes it easy for SMEs to test/refine their work.
Cut Engineering-SME back-n-forth. With Codex, your company’s SMEs know exactly what to work on to maximally improve the AI system, and how to do it (no communication with Engineering needed). Without Codex, this requires lots of back-n-forth between AI engineers and SMEs. SMEs do not know how to discover the queries where the AI response performs badly, particularly the subset of these queries where providing a good answer would have the highest ROI.
Track work gets done and is high-quality. With Codex, the work of SMEs is neatly tracked to ensure the quality of their work. Without Codex, documentation work by SMEs tends to be ad hoc, and it is nontrivial to ensure individual productivity and high-quality output.

Can I export the contents of Codex?

Yes, you can use the Export button in the Web Application to get a Data file that contains all of the information from a Codex Project, including all provided Questions/Answers.

Can SMEs instead work in Slack/Email or another platform?

Yes Codex offers Enterprise API connectors that allow your SMEs to work in any major platform rather than the Codex Web Interface.

Do you offer private deployments?

Yes Codex can be privately deployed in your company’s Virtual Private Cloud, such that no data leaves your firewall.

When my RAG app uses Codex, can one user’s query be exposed to another user?

No, Codex never returns queries when it is consulted by your RAG application. The only output from Codex is answers entered into the platform by your SMEs.

What are the options to integrate Codex into my RAG application?

There are many ways Codex can be integrated into a RAG applications, in order to supply answers to queries the RAG application would otherwise poorly handle.

We recommend two integration options:

Integrating Codex as-a-Tool
Integrating Codex as-a-Backup

These are compared in our Concepts page on Integrations.

Other integration options are possible (e.g. integrating Codex as-a-Cache that is checked first before queries are passed to the RAG system). Contact us to learn more: support@cleanlab.ai