Nemo Guardrails is NVIDIA's open source programmable guardrail toolkit. It provides five types of guardrail mechanisms: input/output/retrieval/execution/dialogue. It supports Colang configuration language OpenAI compatible API service LangChain integration and Docker deployment. It is suitable for LLM application scenarios that require security compliance.

NemoGuardrails

Core parameters and statistics

Nemo Guardrails is a programmable security guardrail toolkit for LLM applications. It is open source and maintained by NVIDIA. It is officially positioned as "easily adding programmable guardrails for LLM-based conversational applications." It does not directly provide a chat interface or model reasoning, but is embedded as a middleware layer between applications and LLM, filtering and constraining input, output, dialogue processes, retrieval results, and execution actions through five types of guardrail mechanisms.

Projects	Public Information
Official Positioning	Programmable Guardrail Toolkit for LLM Dialog Applications
Guardrail Type	Input, Output, Dialog, Retrieval, Execution
Configuration Language	Colang 1.0 / 2.0
Deployment form	Python library CLI, Server (Docker)
Open Source License	Apache 2.0
GitHub Stars	~6,700
GitHub Forks	769
Contributors	126
Latest version	v0.23.0 (2026-07-02)
Python version	3.10 / 3.11 / 3.12 / 3.13
Supported platforms	Web (Server API), Desktop (CLI)

Deployment forms: Nemo Guardrails provides three usage methods: Python library integration, independent Server container and CLI. The Python library is suitable for embedding existing code, the Server mode is suitable for microservice architecture isolation, and the CLI mode is suitable for rapid verification during the development period.

Guardrail System: The five types of guardrails correspond to different sections of the LLM application - input layer (user request filtering), output layer (model reply review), dialogue layer (dialog process control), retrieval layer (RAG block-level filtering), and execution layer (tool call verification), forming full-link security coverage from request to response.

Community activity: 6,700+ Stars, 769 Forks, 126 contributors, reflecting that the project has formed a stable open source ecosystem, and the integration of community contributions includes third-party security services such as PolicyAI, CrowdStrike, Cisco, Trend Micro, and Pangea.

User and market recognition

The market value of Nemo Guardrails is reflected in the dual drive of NVIDIA's brand endorsement and the open source community, positioning it in the rapidly growing compliance area of ​​LLM security.

Academic Recognition: This project has an EMNLP 2023 system presentation paper (Rebedea et al., 2023) and has a public citation record in the academic community, indicating that its architectural design has been peer-reviewed.

Community Adoption: GitHub data shows 6,700+ Stars and 126 contributors, with intensive development activity - the latest commit shows that there were still code updates from 16 hours ago. Issues and Pull Requests remain in double digits (111 Issues / 64 PRs), and community issue feedback and repairs are active.

Enterprise Ecosystem Integration: Nemo Guardrails has integrated with a number of security vendors, including ActiveFence, PolicyAI, CrowdStrike AIDR, Cisco AI Defense, Trend Micro Vision One, Pangea AI Guard, GuardrailsAI, Clavata, etc., covering enterprise-level security scenarios such as content auditing, threat detection and PII desensitization.

Prerequisites: The value of Nemo Guardrails depends on whether the user already has or is building an LLM application. If the team has not yet connected to any LLM capabilities, the guardrail toolkit cannot independently generate revenue. It is more suitable for teams that "exist LLM inference links and need to supplement the security governance layer".

Cost advantage

• C-side/Individual: Usually a free version is provided to experience the core functions, and high-frequency use requires a paid package subscription.
• API/Developer: Billed by call volume, suitable for development teams that can be flexibly integrated into their own systems.
• Enterprise/Privatized: Contact the business owner for customized quotation and deployment plan. The specific price is subject to the official real-time pricing page.

Main functions

The core capabilities of Nemo Guardrails revolve around "five types of programmable guardrails + Colang configuration language + third-party security integration":

• Input Rails: Performs auditing before user requests reach LLM, which can reject input (if a jailbreak prompt is detected), desensitize sensitive data (PII replacement), and rewrite input content. Supports multiple detection methods such as regular matching, classifier model API third-party services, etc.
• Output Rails: Post-review of responses generated by LLM, allowing you to reject output (e.g. harmful content is detected), rewrite responses (e.g. remove sensitive data), and fact-check. 0.23.0 adds context expansion detection, which can intercept excessively long/duplicate/low-entropy/padding content.
• Dialog Rails (Dialog Guardrail): Define dialogue flow constraints through Colang language, control LLM to respond within the preset path, and prevent deviating from the topic or entering unauthorized topics. Supports status management of multi-round conversations.
• Retrieval Rails: In RAG scenarios, retrieved document chunks are reviewed chunk by chunk, allowing specific chunks to be rejected or rewritten before being sent to LLM, preventing sensitive information from being included in the build context.
• Execution Rails: Verify the calling parameters and return values ​​of tools/functions to ensure that external actions called by LLM are executed within the safety boundary.
• Colang Configuration Language: A domain-specific language with Python-like syntax for defining guardrail rules, conversation flows, and user intent. Both Colang 1.0 (default) and Colang 2.0 are supported.
• Guardrails Server: OpenAI compatible HTTP API service, supports /v1/chat/completions, /v1/models, /v1/checks and other endpoints, and can be used as a direct replacement layer for existing applications.
• Built-in guardrail library: Preset guardrails such as jailbreak detection, content security, topic security, fact checking, hallucination detection, PII desensitization, etc. can be activated directly.

Model and version evolution

Continuous iterative updates, the latest version introduces performance optimization and new features. Historical version information can be viewed on the official release page. There is no complete public version evolution timeline yet. It is recommended to pay attention to the official announcement to understand the rhythm of feature updates.

Technical advantages

The technical differentiation of Nemo Guardrails comes from its architectural design of "layered guardrails + programmable process control + third-party ecological integration":

Five-layer guardrail full link coverage: Input → Dialog → Retrieval → Execution → Output The five-layer guardrail is connected in series, and each layer can independently configure the interception/modification/release strategy. This layered design avoids the blind spots of a single detection point - for example, the output guardrail alone cannot prevent jailbreak attempts in user input, while the three-layer combination of input + dialogue + output can form a more complete defense surface.

Colang Programmable Conversation Control: Unlike other guardrail solutions that only provide fixed categories of moderation, Colang allows developers to define custom conversation flows, user intents, and bot behaviors with a Python-like DSL. This makes the guardrail no longer just a "good/bad" binary classification, but can design complex multi-round interaction constraints (such as must authenticate first, ask questions later).

IRails parallel engine: The IORails engine introduced from v0.21.0 supports the parallel execution of input guardrails and LLM inference (i.e. speculative generation). In non-streaming mode, the delay of the input guardrails is hidden within the LLM response time, reducing the end-to-end perceived delay. v0.22.0 extends parallelization capabilities to streaming output guardrails.

Optional dependency architecture: LangChain is downgraded from core dependency to optional starting from v0.22.0, with built-in httpx-based OpenAI compatible client. This change reduces the risk of dependency conflicts and allows scenarios that do not require LangChain to get a more lightweight deployment package. v0.23.0 further reduces the size of the wheel by about 10 times.

Inference model compatibility: v0.20.0+ supports OpenAI inference models (such as o-series), automatically handles compatibility differences in parameters such as temperature, stop, max_tokens, etc., so that the guardrail layer is not affected by changes in the underlying model type.

How to use

Nemo Guardrails offers three usage paths, suitable for teams of different maturity levels:

How to use	Suitable for scenarios	Features
Python library (pip)	Existing LLM application code, need to embed guardrails	Minimal changes, replace the original LLM call through LLMRails.generate()
Guardrails Server (CLI/Docker)	Microservice architecture, hope guardrails can be deployed independently	OpenAI compatible API, can be transparently accessed as a proxy layer
Docker container	Production boundary isolation, resource management and control	Complete Dockerfile, can bind configuration files and custom actions

Python library quick access example:

from nemoguardrails import LLMRails, RailsConfig

# Load guardrail configuration
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)

# Call LLM through the guardrail layer
completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)

Server startup example:

# Start guardrail server
nemoguardrails server --config PATH/TO/CONFIGS --port 8000

# Use OpenAI compatible API
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "config_id": "sample",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Typical guardrail configuration file structure (config.yml):

models:
  - type: main

    engine:openai
    model: gpt-4

rails:
  input:
    flows:
      - check jailbreak
      - mask sensitive data on input
  output:
    flows:
      - self check facts
      - self check hallucination

Implementation is usually carried out according to "first partial and then comprehensive": first enable input + output guardrails on 1-2 low-risk LLM capability points (such as summary generation, content classification), verify the false positive rate and delay impact, and then gradually expand to dialogue processes and high-risk scenarios (such as user identity authentication, payment-related interactions). You need to pay special attention to the false positive rate of guardrails when first accessing - excessive interception will significantly affect the user experience.

Product Pricing

Nemo Guardrails itself is an open source project, licensed under the Apache 2.0 license, with no software purchase fees. The cost is mainly reflected in the reasoning layer and operation and maintenance layer:

• C-side/individual developers: pip install nemoguardrails to use all core functions and built-in guardrail libraries for free. Additional LLM inference costs (models called by guardrails) are borne by the developer.
• Developer/Self-hosted: No licensing fees, infrastructure costs (GPU/CPU servers, storage, network), operation and maintenance manpower, custom guardrail rule development costs. It is recommended to prepare at least 2-4 weeks for guardrail configuration tuning and false alarm rate convergence.
• Enterprise Production Deployment: If you use the NVIDIA Build endpoint for guardrail inference, you need to refer to the official real-time pricing page; there are no external inference fees when fully self-hosted. Enterprises need to additionally consider terms such as SSO integration and audit log SLA, which need to be confirmed with the NVIDIA business team.

Application scenarios

The applicable scenarios for Nemo Guardrails are concentrated in areas where LLM applications already exist or are being built and security compliance governance is required:

• Customer Support and Conversation Bot: Deploy input guardrails (to filter malicious/jailbroken requests) and output guardrails (to ensure that responses comply with company policies and do not contain sensitive information) in customer service AI to reduce manual review and interception costs. Verification focus: The impact of guardrail false alarm rate on the first ring duration.
• RAG Knowledge Base Q&A: Enable search guardrails to conduct block-by-block security audits of checked document blocks to prevent employees from accidentally exposing confidential information during internal knowledge base queries. Verification focus: The impact of block-level guardrails on retrieval recall rate.
• Financial compliance and legal review: In high-risk scenarios such as contract review and compliance Q&A, full-link guardrails (input + dialogue + output) are enabled, and third-party security APIs (such as ActiveFence and PolicyAI) are used for multi-level filtering. Verification focus: end-to-end delay after multi-layer guardrails are connected in series.

Applicable people

• LLM application developers: Teams that need to add a security layer to their self-built AI applications can quickly integrate through Python libraries or Server. Colang learning costs approximately 1-3 days.
• AI Security and Compliance Engineer: The role responsible for formulating and implementing AI application security policies. Colang can be used to encode security policies into executable guardrail rules to replace manual review of SOPs.
• Platform Architect/SRE: Teams that need to unify the governance of the LLM security layer can use Guardrails Server to achieve architectural decoupling of guardrails and business logic.

Not suitable for the crowd: Teams that do not use any LLM capabilities (guardrails are meaningless); individual users who require "out-of-the-box, zero-configuration" security solutions (requiring a certain amount of investment in development and configuration); scenarios that are extremely sensitive to end-to-end latency (each guardrail inference adds 200-1000ms, depending on the model and deployment method).

Summary and Outlook

Nemo Guardrails provides a customizable, auditable, and scalable middleware solution for LLM application security management through a combination of five types of programmable guardrails + Colang configuration language + open source license. It is not a "one-button-on" safety switch, but a guardrail framework that requires investment in configuration and tuning. The intensive iteration from v0.14.1 to v0.23.0 (approximately 4-6 Monday releases) shows NVIDIA's continued investment in this field, especially the IORails parallel engine and LangChain decoupling, which reduces latency and dependency risks in actual deployment.

Current limitations and uncertainties: There is no public benchmark data for the false positive rate (false positive) and false negative rate (false negative) of the guardrail, and the team needs to conduct stress testing on its own in the target scenario; the applicability of the built-in guardrail library needs to be evaluated in conjunction with actual business scenarios, and the official document clearly states that "built-in guardrails may not be applicable to all production scenarios"; the production authorization terms and prices of NVIDIA Build endpoints have not yet been publicly and stably displayed; Colang 2.0 is still in the functional improvement stage, and it is recommended to use Colang 1.0 for production.

Procurement and Adoption Suggestions: It is recommended that new teams first complete the PoC within 1-2 weeks through pip install nemoguardrails + official sample configuration, focusing on evaluating whether the guardrail reasoning delay, false positive rate and Colang configuration meet the complexity of the business rules. Enterprises need to confirm with the NVIDIA commercial team before purchasing: the NVIDIA Build license terms in the production context, the architecture support boundaries of private deployment, and the compatibility of LangChain integration on the target LLM provider.

Related tools: GitHub Copilot, Cursor

Version evolution of Nemo Guardrails

Judging from the public information of GitHub Releases, the version iteration rhythm of Nemo Guardrails is about 4-6 Mondays, and the main line of evolution from 0.14.1 to 0.23.0 can be summarized into three stages:

Basic capability building period (v0.14.1 - v0.17.0)

• v0.14.1 (2025-07-09): Jailbreak detection configuration simplified LLMRails constructor fix.
• v0.15.0 (2025-08-08): Parallel guardrail execution, streaming output guardrail OpenTelemetry tracing infrastructure.
• v0.16.0 (2025-09-05): OpenTelemetry semantic convention tracking GuardrailsAI integrated with Pangea community.
• v0.17.0 (2025-10-10): Tool calls pass through RunnableRails rewrite, tool input/output guardrails.

Engine reconstruction and ecological expansion period (v0.18.0 - v0.20.0)

• v0.18.0 (2025-11-07): BotThinking inference trace guardrails, multi-embedded provider Cisco AI Defense integrated LFU cache.
• v0.19.0 (2025-12-04): LangChain v1.x compatible.
• v0.20.0 (2026-01-22): Inference content security model GLiNER PII detection, multi-language reject messages.

IORails and optimization period (v0.21.0 - v0.23.0)

• v0.21.0 (2026-03-12): IORails optimization engine (parallel non-streaming guardrail), OpenAI compatible Server, LangChain Agent middleware.
• v0.22.0 (2026-05-23): IORails streaming output guardrail OTEL tracing LangChain decoupled into optional, inference model support.
• v0.23.0 (2026-07-02): Hugging Face classifier guardrails, context bloat detection, Polygraf PII detection, NumPy search replacement Annoy, wheel package reduced by 10 times.

Version Info

• Nemo Guardrails 0.23.0 ：Added Hugging Face classifier guardrails, context expansion detection, Polygraf PII detection and desensitization; NumPy precise search replaces Annoy; tool calls support streaming and non-streaming requests; the wheel package size is reduced by about 10 times. （2026-07-02）
• Nemo Guardrails 0.22.0 ：IORails supports streaming output guardrails and OpenTelemetry tracing; LangChain is decoupled as an optional dependency; inference model support; Azure is preset as a first-class default framework. The official real-time page shall prevail. （2026-05-23）
• Nemo Guardrails 0.21.0 ：Introducing IORails optimized input/output guardrail engine to support parallel execution; OpenAI compatible guardrails server; LangChain Agent middleware integration; PolicyAI, CrowdStrike AIDR community integration. The official real-time page shall prevail. （2026-03-12）
• Nemo Guardrails 0.20.0 ：Inference content security model support; GLiNER open source PII detection; multilingual deny messages; streaming configuration simplified. The official real-time page shall prevail. （2026-01-22）
• Nemo Guardrails 0.19.0 ：LangChain v1.x compatible support. The official real-time page shall prevail. （2025-12-04）
• Nemo Guardrails 0.18.0 ：BotThinking inference trace guardrails; Azure OpenAI/Cohere/Google embed providers; Cisco AI Defense integration; LFU in-memory cache. The official real-time page shall prevail. （2025-11-07）
• Nemo Guardrails 0.17.0 ：Tool call passthrough support; RunnableRails rewrite; tool input/output guardrails; Trend Micro community integration. The official real-time page shall prevail. （2025-10-10）