LMQL (Language Model Query Language) is a special programming language for large language models that allows developers to use declarative syntax to control the output format, constraints and reasoning process of LLM. It supports type-safe structured output, multi-step reasoning chains, branching logic and constraint decoding, transforming prompt word engineering from a "black box experiment" to a "programmable deterministic process".

LMQL

Core parameters and statistics of LMQL

LMQL (Language Model Query Language) is not another LLM packaging library, nor is it a prompt word template engine - it is a complete programming language that uses declarative syntax to transform LLM calls from "black box experiments" to "programmable deterministic processes." It was developed by the ETH Zurich Safe and Reliable Intelligence Laboratory (SRI Lab) and is aimed at developers and researchers who need precise control of LLM output.

Projects	Public Information
Official positioning	A programming language for large language models
Core technology types	Agent/MCP/automation tools - control LLM behavior through language-level constructs
License Agreement	Apache-2.0 Open Source
Installation form	Python library (pip install lmql), cross-platform
Home	DE (ETH Zurich, SRI Lab)
GitHub Stars	~4,200
GitHub Forks	~221
Contributors	~35
Latest release	0.7.3 (2023-10-15)
Supported backends	OpenAI, Azure OpenAI, HuggingFace Transformers, llama.cpp, Replicate

A brief comment: LMQL solves the core contradiction in the Prompt project - developers want to use deterministic logic to control output, while LLM is naturally probabilistic. It introduces constraint decoding, type system and control flow at the language level, making LLM calls written like Python and run with boundaries.

Status Reminder: The main active development period of LMQL will be in 2023, and the latest stable version 0.7.3 will be released in October 2023. Subsequent project updates will focus on community maintenance and document improvement, and no major new versions will be released. This means that it is forward-looking in terms of technical concepts, but when used as a production dependency, the long-term maintenance activity of the project needs to be evaluated.

Users and market recognition of LMQL

The influence of LMQL is mainly reflected in the academic research community and early LLM engineering practitioner group, rather than large-scale commercial adoption.

Academic Endorsement: Produced by the ETH Zurich SRI Lab, the research background provides the theoretical depth to the project. LMQL's constraint decoding idea directly influenced the design of multiple subsequent LLM tool chains (including LangChain's partial output parser Guidance and other similar projects). Relevant academic papers and blog posts have stable citations in the NLP/ML community.

Community size: About 4,200 Stars, 221 Forks, and 35 contributors on GitHub. It is a "high-profile, medium-sized" open source project. Its Discord community and technical blog (lmql.ai/blog) record detailed version evolution and design decisions, which are of reference value for researchers and in-depth users.

Industry Impact: The concept of "controlling LLM with programming language" proposed by LMQL was a cutting-edge concept in 2023, and was subsequently adopted or used for reference by many companies and open source projects. However, LMQL itself has not moved towards large-scale commercial applications, and its value is more reflected in "technical thought output" rather than "user growth". If activity and user scale are used as criteria, LMQL is still in the "influential research prototype" stage rather than a mature production tool.

Cost advantage of LMQL

The cost structure of LMQL is very simple: the language itself is completely open source and free, and the cost comes from the pay-as-you-go billing of the connected LLM backend.

• C-side/Personal: LMQL is a Python library, completely free (Apache-2.0 license). Individual developers can install and use it locally unlimited times without any subscription fees. However, if you use a cloud backend such as OpenAI, you need to bring your own API Key and pay according to the backend pricing.

• API/Developer: LMQL itself has no API call fees. Developers only need to pip install lmql and then configure the credentials of the target LLM backend (such as OpenAI API Key). LMQL acts as a "translation layer + constraint engine" at runtime - it does not incur a separate API fee, but compiles queries into calls to the LLM backend. This means that the Token consumption of LMQL directly affects the back-end bill: the constraint decoding and caching layer can significantly reduce Token consumption (official data says that the caching layer can reduce Token usage by 33%-80%), thereby indirectly reducing the back-end API billing.

• Enterprise/privatization: Enterprises can download the LMQL source code for free for private deployment or secondary development (Apache-2.0 license). For data-sensitive scenarios, it can be combined with a local HuggingFace model or llama.cpp to run completely offline without any external API calls. The cost at this time is mainly GPU server purchase/lease costs and operation and maintenance manpower. LMQL's tree cache layer can reuse cache results in enterprise batch inference scenarios, further reducing the computing power overhead of repeated queries.

Cost Dimension	C-side/Individual	API/Developer	Enterprise/Privatization
LMQL License Fee	Free	Free	Free
LLM back-end cost	Bring your own Key, pricing is based on back-end	Bring your own Key, pricing is based on back-end	Build your own model or pay as you go
Infrastructure	PC only	Rely on backend API	GPU server + operation and maintenance
Hidden costs	Learning LMQL syntax	Constraint decoding and debugging	Cache management, version compatibility

Main functions of LMQL

The capabilities of LMQL are not a platter of "one tool with multiple functions", but a collaboration of four layers of capabilities through language design: constraints + control flow + decoding algorithm + back-end abstraction. Each layer can be used independently, but its true value is revealed when combined.

• Constraint Decoding: The core differentiation capability of LMQL. Declare output constraints through the where clause, such as len(TOKENS(ANSWER)) < 120, STOPS_AT(ANSWER, "."), INT(NUM), REGEX(RESPONSE, r"[0-9]{2}/[0-9]{2}"). Constraints are forced to take effect through logit masking during the decoding phase, rather than string regular matching after generation. This means that the model will not produce illegal output at all when constraints are not hit - this is essentially different from "post-processing verification" and reduces the number of retries and token waste.

• Type-safe structured output: LLM output can be directly constrained to be a valid structured object via type(VAR) is Person (where Person is a Python dataclass). LMQL automatically translates type definitions into decoding constraints, ensuring that the output can be correctly parsed as Python objects. This is especially valuable when extracting JSON-formatted data from unstructured text—no need to write complex output parsers, or harp on the output format in the prompt.

• Nested Queries and Procedural Prompt Programming (Nested Queries): The "Procedural Prompt Programming" capability introduced in version 0.7. Developers can encapsulate prompt logic as a @lmql.query function and call it like a normal function in the top-level query. For example, define a chain_of_thought function to do chain of thought reasoning and call it in the top-level [ANSWER: chain_of_thought]. Nested queries automatically execute the process of "instruction injection → generation → instruction removal", which is similar to function calling and stack expansion in traditional programming.

• Multiple decoding algorithm support: Supports multiple decoding strategies such as argmax (greedy decoding), sample(temperature=1.2) (sampling decoding), beam(N) (beam search) and best_k. Developers can switch decoding methods at different stages of the same query, for example, first use sample for exploratory generation, and then use argmax for deterministic output.

• Caching Layer: A tree cache structure that caches all tokens, logits and metadata output by LLM. In the template multi-variable scenario, the token consumption can be reduced by 77% and the number of requests by 75%; in the long constraint short-circuit scenario, the token consumption can be reduced by 80%; in the tool enhancement scenario, the number of interactions can be reduced by 33%. The cache can be persisted to disk and reused across queries.

• Tool Enhancements (Actions): Preview feature that allows LLM to call arbitrary Python functions (such as wiki(q), calc(expr)) during inference. The calling protocol is handled automatically by the LMQL runtime and developers only need to declare the available tools in inline_use(REASONING, [wiki, calc]).

LMQL model and version evolution

The version history of LMQL clearly shows the evolutionary path from "academic prototype" to "fully functional language". April to October 2023 is an intensive iteration period, after which the project enters a stable maintenance state.

Early Groundbreaking (2023-04 to 2023-06)

• LMQL 0.0.5 (2023-04-17): Early stable version, focusing on performance optimization and stability improvements. The first batch of community contributions are merged, marking the project's transition from a single-person research prototype to a community collaborative project.
• LMQL 0.0.6 (2023-05-01): Milestone version - introduction of caching layer. Tree structure caching reduces the token consumption of template queries by 77% and long constraint scenarios by 80%. This is LMQL’s most critical innovation in engineering efficiency.
• LMQL 0.0.6.1 (2023-05-03): Cache layer bug repair and optimization, new HTTP/WebSocket/SSE output writer, so that LMQL can be embedded in web services.
• LMQL 0.0.6.3 (2023-05-11): Lighter runtime (removes mandatory dependency on transformers), new TOKENS(...) constraint function and conditional stopping capability.
• LMQL 0.0.6.4 (2023-06-08): Major engineering improvements - Azure OpenAI supports the LMTP protocol to speed up local model inference by 5-6 times, and the synchronous Python API simplifies the use of the tiktoken word segmentation backend.

Syntax simplification and multiple backends (2023-07)

• LMQL 0.0.6.5 (2023-07-14): Syntax "minimization" reform. LMQL code is closer to standard Python, and queries can be defined in one line; new @lmql.query decorator and lmql.F lambda function are added; new llama.cpp backend and inline constraints are added. This refactoring has significantly lowered the threshold for getting started.
• LMQL 0.0.6.6 (2023-07-25): lmql.F supports positional parameters, improves llama.cpp error handling, supports auto_gptq quantization model. Community contributions increased significantly.

Function explosion (2023-10)

• LMQL 0.7 (2023-10-10): The largest update and the last major version. Introducing nested queries (procedural prompt programming), Generations API (lightweight generation + scoring interface), Chat API (one-click deployment of chatbots), reasoning certificates (reproducible reasoning records), variable decorators, and multi-backend extensions (Replicate, sentencepiece). LMQL Actions (tool calls), regex constraints and type/data class constraints are also released in preview form.
• LMQL 0.7.1 (2023-10-12): Fix the compatibility issue between distribution clause and inference tracing.
• LMQL 0.7.2 (2023-10-13): Ensure lmql playground command is available in PyPI package.
• LMQL 0.7.3 (2023-10-15): Fixed an issue where Chat API resource files were not included in the PyPI package.

Since then, the project has not released major version updates, mainly focusing on document improvement and community maintenance.

Technical advantages of LMQL

The technical difference of LMQL is not "more functions", but "replacing prompt parameters with language design" - it elevates LLM engineering from prompt word experiments to the abstraction level of a programming language.

Mechanism and effect of constraint decoding: The traditional approach is to write "Please output JSON format" in the prompt, and then parse and retry through post-processing. LMQL directly masks illegal tokens through logit masking at the decoding layer - if the constraint requires the output of integers, the probability of non-numeric tokens is directly set to zero when the language model is generated at each step. This brings two effects: ① Output compliance changes from "high probability" to "deterministic"; ② Post-processing and retry logic are not required, reducing Token waste. Constraint short-circuiting is further optimized: once the model outputs the result of a determined constraint (such as "Option A" is selected), the remaining tokens are automatically completed by the constraints without calling LLM.

The engineering value of the tree cache layer: In traditional LLM calls, multi-variable templates require multiple independent requests - each request contains the same context prefix, resulting in a large amount of token and delay waste. LMQL's tree cache stores all candidate branches (logits, tokens, metadata) for each token position as reusable nodes. When the same template is executed, if the output of LLM is aligned with the template, the variables are filled directly without re-calling - official data can reduce the number of requests by 33%-80%. The cache can be persisted to disk, which is particularly useful during the query development phase: when iterating the same query multiple times, the cache automatically hits the unchanged parts and only calls are made for the newly changed parts.

Backend Abstraction Layer: LMQL is not a language that binds specific models. The same piece of LMQL code can be switched between backends by changing the from clause - OpenAI, Azure, HuggingFace Transformers, llama.cpp, Replicate are all supported. This means that you can use lightweight models for debugging during the development phase and switch to larger models during the production phase without changing the query logic. However, it should be noted that different backends have different levels of support for constraint decoding: OpenAI’s Completions API supports logit bias, while Chat API has limited support for constraints. This is a limitation that needs to be considered when selecting.

How to use LMQL

The usage path of LMQL is divided into three steps: "local installation → writing query → running and debugging", covering different needs from quick early adopters to production integration.

How to use	Suitable for the crowd	Features	Cost
Local installation (pip)	All developers	pip install lmql, runs completely locally	Free
Playground IDE	Quick experience	Browser side lmql.ai/playground, zero installation	Free
API Integration (Python)	Application Developers	Embed into existing Python projects	Free (billed per backend API)
Privatization + local model	Data compliance scenario	HuggingFace or llama.cpp local inference	Free (GPU computing power cost)

Quick Installation with Hello World:

pip install lmql

After installation, you can use the lmql playground command to start the browser IDE, or directly write a Python file to run.

importlmql

# The simplest LMQL query: declarative constraints
@lmql.query
def hello():
    '''lmql
    "Say 'this is a test':[RESPONSE]"
    where len(TOKENS(RESPONSE)) < 25
    '''
    return RESPONSE

print(hello())

Configuring the OpenAI backend: If you use the OpenAI model, you need to set the context variable OPENAI_API_KEY, or create the api.env file in the current directory:

openai-org: <org identifier>
openai-secret: <api secret>

It is also possible to use the LMQL-specific context variables LMQL_OPENAI_SECRET and LMQL_OPENAI_ORG.

Run LMQL program:

• lmql playground: launches the browser IDE (requires Node.js), including sample display and debugging panel.
• lmql run <file>.lmql: Execute local .lmql file.
• lmql serve-model: Start the inference API service of the local HuggingFace model (this command must be executed first when using the local model).
• Python integration: embed LMQL queries into standard Python code via the @lmql.query decorator.

Structured Output Example (type constraint preview feature):

importlmql
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int
    job: str

@lmql.query
def extract_person():
    '''lmql
    "Alice is a 21 years old engineer at LMQL Inc.\n"
    "Structured: [PERSON_DATA]\n"
    where type(PERSON_DATA) is Person
    '''
    return PERSON_DATA

result = extract_person()
print(result) # Person(name='Alice', age=21, job='engineer')

API Integration: LMQL 0.7’s Generations API provides a lightweight generation and scoring interface without the need to write full LMQL queries:

importlmql

m: lmql.LLM = lmql.model("openai/gpt-3.5-turbo-instruct")
result = m.generate_sync("Hello", max_tokens=10)
print(result) # "Hello, I am a 23 year old female."

Product Pricing for LMQL

LMQL itself is completely open source and free (Apache-2.0 license). The real difference in cost structure comes from the LLM backend plugged in:

C-side/Individual: Zero fee. After installing LMQL locally, if you are using HuggingFace's local model or llama.cpp, there is no API fee and all you need is a PC to run. If you use cloud backends such as OpenAI and Azure, you need to bear the API call fees. LMQL's caching layer can help reduce Token consumption and indirectly reduce back-end costs.

Developer/API Integration: No call billing for the LMQL library itself. After developers integrate LMQL in their applications, Token consumption directly determines the back-end bill. Taking OpenAI gpt-3.5-turbo-instruct as an example, LMQL's constrained short-circuit and tree cache can reduce token usage by 33%-80%, which is the core economic incentive for using LMQL instead of directly calling the API.

Enterprise/Private: LMQL's Apache-2.0 license permits any commercial use, including modification and redistribution. Enterprises can deploy the combination of LMQL + local models completely offline without paying API fees to any third party. The cost structure at this time is: GPU server purchase/lease (such as A100, H100), electricity, operation and maintenance personnel, and the license of the model itself (depending on the selected model, open source models such as Llama are free, enterprise-level models may require additional licenses).

Taken together, the economic value of LMQL does not lie in "whether it is expensive to use LMQL", but in "how much the cost of LLM calls can be reduced by using LMQL" - this is a question that requires measurement in specific scenarios.

Application scenarios of LMQL

LMQL's capabilities focus on technical scenarios that require precise control of LLM output, reduced post-processing costs, and improved output reliability.

• Structured Data Extraction: Extract structured data in JSON format from unstructured text (emails, reports, chat logs). LMQL's type constraints ensure the correct format of the output fields without regular post-processing. Implementation Tip: In regulatory compliance scenarios, you can first use type constraints to constrain the output format, and then combine the inference certificate to record the complete inference link of each extraction for audit traceability.

• Batch LLM pipeline and ETL tasks: In pipelines that require classification, summarization, and entity extraction of large batches of text, LMQL's caching layer can significantly reduce repeated Token consumption. For example, performing sentiment classification on 10,000 customer service conversations - LMQL's tree cache will reuse the LLM output of the query prefix, reducing the number of API calls by about 50%. Implementation Tips: Verify the correctness of the constraints on a small sample first, and then run it in full to avoid batch failures caused by incorrect constraint definitions.

• Agent and Tool Enhanced Prototype: LMQL Actions (preview) allow LLM to call external functions (such as searches, calculations, database queries) during inference. Although this feature is in the preview stage, it provides a low-cost exploration method for the technical team to verify the feasibility of "Agent-style LLM application". Implementation Tip: The preview feature is unstable and is not recommended for use in production-level Agent systems. It can be used as a proof-of-concept tool in the design prototype stage.

• LLM Behavioral Research and Experiments: The @distribution clause of LMQL can obtain the probability distribution at the token level, and the inference certificate can record the complete inference context and parameters. For NLP researchers and Prompt engineers, this is a useful tool for analyzing LLM output behavior, testing the effects of constraints, and comparing different decoding strategies. Implementation Tips: The inference certificate function was introduced in version 0.7. It is a stable feature and is suitable for paper experiments and Prompt effect comparisons.

Applicable groups of LMQL

LMQL's audience is concentrated among developers and researchers with strong technical capabilities - it is not an "out-of-the-box" tool, but a set of programming languages that require investment in learning costs.

• LLM Application Developers: For developers who are building LLM applications that require structured output or multi-step reasoning, LMQL's language-level constraints can significantly reduce post-processing code and prompt debugging time. Prerequisites: Requires familiarity with Python syntax and basic LLM calling concepts.

• AI/ML researcher: An academic researcher who studies LLM output behavior and constraint decoding effect Token probability distribution. LMQL's constraint language and inference certificates provide a standardized experimental context. Prerequisite: There is a need to understand the theoretical background of LMQL (logit masking, beam search decoding).

• Prompt Engineer: Practitioners who need to frequently test prompt constraint boundaries and compare different decoding strategies. LMQL's Playground IDE and caching layer provide more efficient iteration than manual API calls. Precondition: You need to understand the difference between declarative syntax and imperative syntax.

• Not suitable for the crowd: ① Scenarios that only require simple dialogue or text generation and do not have strict requirements on the output format - LMQL's constraint capability adds unnecessary syntax overhead; ② Real-time applications that are extremely sensitive to latency - the logit masking calculation introduced by constraint decoding will add additional latency; ③ Teams that prefer low-code/no-code tools - LMQL is essentially a programming language and requires writing code; ④ Use OpenAI Chat Scenarios where models (gpt-3.5-turbo, gpt-4) serve as the main backend - OpenAI's Chat API has limited support for logit bias, and some LMQL constraints cannot fully take effect.

Summary and Outlook of LMQL

LMQL’s core competencies and current limitations are very clear. The concept of "controlling LLM with programming language" proposed by it in 2023 is still forward-looking today, and its design of constraint decoding and tree cache directly affected multiple subsequent LLM tool chains. However, for the actual selection in 2026, the current activity level of the project needs to be taken into consideration.

Core Value: LMQL is one of the very few projects that solves the controllability problem of LLM at the "language level" rather than the "library level". Its constraint decoding mechanism is technically superior to prompt engineering and post-processing solutions, and the Token saving effect of the cache layer has verifiable quantitative data. For structured output-intensive LLM applications, LMQL can significantly reduce development complexity and running costs.

Current limitations: ① The main active development period of the project remains in 2023, and the latest version 0.7.3 is nearly three years old. It lacks native adaptation and performance optimization for new models (such as the GPT-4 series Claude series Gemini series); ② OpenAI Chat API's restrictions on logit bias cause some of LMQL's constraint capabilities to be unavailable on this backend; ③ The preview features (Actions, type constraints) remain in the experimental stage and have not moved towards stability; ④ The community size is limited (4.2k Stars), and third-party integration and ecological support are far less than mainstream frameworks such as LangChain and LlamaIndex.

Procurement/Adoption Risk Assessment: If you consider adopting LMQL in production in 2026, you need to focus on evaluating the following points: ① Project Activity - It is recommended to check GitHub's commit and issue response frequency in the past 6 months to confirm whether community maintenance meets the funding requirements for production dependencies; ② Model Compatibility - Verify the compatibility of the target LLM backend (especially the Chat model) with LMQL constraint decoding, which can be done through the Playground Do a small-scale test first; ③ Comparison of alternatives - Guidance (Microsoft), Outlines, Instructor and other similar projects will have more active iterations between 2024 and 2026. It is recommended to benchmark their functional coverage and maintenance activity; ④ Long-term feasibility - If the project has no major updates for a long time, it may be necessary to position LMQL as a "technical reference" rather than a "long-term dependency", and reserve a replacement path in the architecture design. For research prototypes and personal development projects, LMQL remains the best entry point to experience the concept of "constraint decoding".

Tool open list (LMQL syntax and language constructs)

LMQL does not expose a RESTful Tool interface in the traditional sense, but provides precise control of LLM behavior through language-level constructs. The following are the core syntax constructs that can be called directly in LMQL programs:

• argmax / sample(temperature=1.2) / beam(N) / best_k: Control the decoding strategy. argmax is a deterministic greedy decoding, sample introduces randomness, and beam and best_k use beam search to explore multiple generation paths.
• where constraint clause: The core language-level constraint mechanism. Supports len(TOKENS(VAR)) < N (Token length constraint), STOPS_AT(VAR, ".") (stop phrase), INT(VAR) (integer constraint), REGEX(VAR, r"...") (regular constraint), type(VAR) is DataClass (type constraint).
• @lmql.query decorator: marks the function as an LMQL query. Supports model, temperature, cache and other parameters, allowing LMQL code to call, pass parameters and return values ​​like ordinary Python functions.
• inline_use(VAR, [func1, func2]): Preview feature, exposing external Python functions for LLM to call in the inference loop. LMQL automatically manages the calling protocol and result insertion.
• FOR loop and control flow: LMQL is a superset of Python and supports standard control flows such as for and if/else. Template variables can be dynamically inserted into the loop to achieve dynamic length generation.
• @distribution: Obtain the token-level probability distribution, which is used to analyze the generation preference of LLM.
• @decorator: Custom variable decorator function, which can perform streaming conversion on model output (such as capitalization, formatting, type conversion).
• Nested query [VAR: query_func]: Call another @lmql.query function as a subquery in the top-level query to automatically perform instruction hiding and result extraction.

Architecture link

LMQL source code (.lmql / @lmql.query)
    │
    ▼
LMQL parser
    │ (parse LMQL syntax into intermediate representation)
    ▼
LMQL core interpreter
    │ (Manage control flow, variable status, constraint registration)
    ▼
约束编译器
    │ (Compile where clause into logit mask / token filter)
    ▼
后端适配器层
    │
    ├── OpenAI Adapter → OpenAI Completions/Chat API
    ├── Azure Adapter → Azure OpenAI API
    ├── HF Adapter → HuggingFace Transformers (local/remote)
    ├── llama.cpp adapter → llama.cpp C++ inference engine
    └── Replicate Adapter → Replicate Cloud Inference
    │
    ▼
Constraint Decoder (Logit Masking)
    │ (apply constraint filter before each decoding step)
    ▼
Tree-based Cache
    │ (Cache tokens, logits, metadata, support branch reuse)
    │
    ▼
Multi-step generation/feedback loop
    │
    ▼
Structured output/Python variables

Control flow direction: LMQL source code → (parse → execution → constraints → decoding) → LLM backend → (token flow → constraint verification → cache) → Python output. Data return path: The token stream output by LLM is filtered by the constraint decoder, recorded by the tree cache layer, and finally mapped to the value of a Python variable.

工程踩坑指南

1. Dead-end loop and Token inflation control: LMQL supports FOR loop and conditional branching, but if the constraints are not properly defined (such as the stop phrase does not match or the length constraint is too loose), it may cause LLM to generate unlimited tokens. Solution: Always set a hard upper limit of len(TOKENS(VAR)) < N in the where clause; set a reasonable max_tokens when using sample to decode; for multi-step inference chains, set a limit on the total number of steps in the outer Python code.

2. OpenAI Chat API constraint compatibility: LMQL's constraint decoding is implemented through logit masking, but OpenAI's Chat Completion API has limited support for logit_bias (only bias adjustment of up to 20 tokens is supported), resulting in complex constraints not fully taking effect on the gpt-3.5-turbo or gpt-4 series models. Solution: Prioritize using OpenAI's Completions API model (such as gpt-3.5-turbo-instruct) or HuggingFace local model to obtain complete constraint decoding support; if you need to use the Chat model, position LMQL as "prompt word arrangement" rather than "constraint enforcement".

3. Cache Expansion and Memory Management: The tree cache is an append-only structure and will continue to grow over a long period of time, possibly causing memory overflow. Solution: Disable caching (cache=False) in long-running queries; use persistent cache files to manage by session; set a periodic cache cleaning policy for production-bound applications.

4. Differences in constraint behavior across backends: The decoding behavior of the same piece of LMQL code on different backends may be inconsistent - the coverage and accuracy of constraints on HuggingFace is higher than on OpenAI. Solution: Complete backend locking during the development phase, and do not frequently switch backends in the production environment; if multiple backends must be supported, write independent query test cases for each backend.

Get started quickly in 3 minutes

# Install LMQL (requires Python 3.10)
pip install lmql

# 验证安装
lmql --help

# hello.lmql or use it directly in Python
importlmql

# Method 1: @lmql.query decorator
@lmql.query
def greet():
    '''lmql
    "Greet LMQL:[GREETING]\n"
    where STOPS_AT(GREETING, ".") and not "\n" in GREETING
    '''
    return GREETING

print(greet())

# Method 2: lmql.run_sync (synchronization API)
program = """
argmax
    "The capital of France is:[ANSWER]"
from
    "openai/text-davinci-003"
where
    STOPS_AT(ANSWER, ".") and len(TOKENS(ANSWER)) < 10
"""
result = lmql.run_sync(program)
print(result)

# Method 3: Start Playground IDE
# Run in terminal: lmql playground
# Browser access: http://localhost:3000

Deployment configuration (LMQL is not exposed through the MCP protocol, but is used as a Python library without a separate server configuration file. For private deployment, you can refer to the official Docker image):

# Use Docker to run LMQL Playground
docker run -p 3000:3000 lmql/lmql

The above code examples are based on the LMQL 0.7.x official documentation and GitHub README. For specific API and syntax details, please refer to the README of github.com/eth-sri/lmql and the official documentation lmql.ai/docs. For actual operation details such as the OpenAI API credential configuration method and the latest tags of the Docker image, please refer to the latest instructions in the official repository.

Version Info

• LMQL 0.7.3 ：Fixed the problem that the resources required by the LMQL Chat API are not included in the PyPI package. This is a maintenance fix version of the 0.7 series. （2023-10-15）
• LMQL 0.7.2 ：Ensure that the lmql playground command is properly distributed as part of the PyPI package. （2023-10-13）
• LMQL 0.7.1 ：Fix minor issues in 0.7, including distribution clause compatibility with inference tracking, automatic chunk_size optimization, etc. （2023-10-12）
• LMQL 0.7 ：The largest update, introducing nested query Generations API, Chat API, inference certificates, decorators, multi-backend support, and preview functions such as Actions, regular constraints, and type constraints. （2023-10-10）
• LMQL 0.0.6.6 ：lmql.F supports positional parameters, improves llama.cpp backend error handling, fixes the LMTP scheduler CPU no-load high usage problem, and supports the auto_gptq quantization model. （2023-07-25）
• LMQL 0.0.6.5 ：Major syntax simplification - LMQL syntax is closer to standard Python, and queries can be written in one line; new llama.cpp inference backend, inline constraints, @lmql.query function decorator lmql.F lambda function. （2023-07-14）
• LMQL 0.0.6.4 ：New Azure OpenAI support, Language Model Transfer Protocol (LMTP) speeds up local model inference by 5-6 times, and synchronous Python API supports tiktoken word segmentation backend Docker image. （2023-06-08）
• LMQL 0.0.6.1 ：Cache layer bug fixes, stop phrase optimization, asynchronous output writer HTTP/WebSocket/SSE output endpoint. （2023-05-03）
• LMQL 0.0.6 ：Introducing the LMQL cache layer to cache tokens and logits based on the tree structure, template query can reduce token consumption by 77% and the number of requests by 75%; constraint short-circuiting can reduce token consumption in long constraint scenarios by 80%. （2023-05-01）
• LMQL 0.0.5 ：An early stable version, focusing on performance optimization and stability improvements, and the first batch of community contributions are merged. （2023-04-17）