LLM Usage "picture_description_api"

[hajaraniwana]

Unable to Get LLM Usage from picture_description_api

Hi , I have a problem when trying to get LLM usage/token usage from the API used by Docling's picture_description_api.

I am using picture_description_api to generate descriptions for images. The image description itself is generated successfully, and I receive a response like this:

{
  "meta": {
    "summary": null,
    "description": {
      "confidence": null,
      "created_by": "not-implemented",
      "text": "The image features a cute, cartoon-style duck with a bright orange head and large, shiny black eyes. It has a small tuft of feathers on top of its head and an orange beak. The duck is holding a white piece of paper with black lines representing text, as if it is reading or presenting something. Its orange arms are wrapped around the paper. The duck is centered within a thick black circular border, creating a logo-like appearance. The overall style is simple, colorful, and friendly."
    },
    "classification": null,
    "molecule": null,
    "tabular_chart": null,
    "code": null
  }
}

The image description is generated correctly, but I cannot find any information about the LLM usage, such as:

• prompt_tokens
• completion_tokens
• total_tokens
• token IDs
• or other LLM usage metadata

For example, when I call my LLM API directly, I receive usage information such as:

{
  "usage": {
    "prompt_tokens": 415,
    "completion_tokens": 619,
    "total_tokens": 1034
  }
}

However, when the LLM is called through picture_description_api, this usage information does not appear to be exposed in the resulting Docling document metadata.

My questions

1. Does picture_description_api currently support returning or exposing LLM usage/token information?
2. If it does, where can I access the usage information from the PictureDescriptionData / picture metadata?
3. If it is not currently supported, is there a recommended way to customize or extend the picture description API to capture the LLM response usage?
4. Is there an example implementation for a custom picture_description_api that preserves the LLM usage information?

I am using a custom LLM API that is compatible with the OpenAI API format, and the API response contains the usage field correctly when called directly.

Thanks!

[dosubot[bot]]

Hi @hajaraniwana! I'm Dosu and I'm helping the IBM Docling team.

PictureDescriptionApiOptions does support capturing LLM usage/token information. This feature was added in PR #3632. If your output only shows confidence, created_by, and text with no usage data, you are likely running a version of Docling from before that PR was merged. Here are the answers to your specific questions:

1. Does it support returning usage information?
Yes. PictureDescriptionApiOptions has a usage_response_key field (default "usage") that tells Docling which key in the API's JSON response holds the usage payload. It supports dotted paths (e.g. "meta.usage") and can be set to None to disable capture.

2. Where to access the usage information?
The usage payload is stored as custom metadata on PictureItem.meta.description, not as a top-level key you'd see in a plain JSON dump. Retrieve it in code like this [1]:

usage = picture.meta.description.get_custom_part()["docling__usage"]

Internally, the namespace "docling" and field name "usage" are joined with __ when stored via set_custom_field().

3. How does the capture work / how to customize?
api_image_request() extracts whatever is at the configured usage_response_key from the raw JSON response before validating it against the OpenAI-compatible schema. If your custom API nests usage somewhere other than the top-level usage key, just point usage_response_key at the correct dotted path:

PictureDescriptionApiOptions(
    url="https://your-api/v1/chat/completions",
    usage_response_key="meta.usage",  # or wherever your API puts it
    ...
)

4. Is there an example implementation?
Yes — docs/examples/picture_description_api_usage.py is a full end-to-end script that converts a PDF, calls an OpenAI-compatible endpoint (including Azure OpenAI), and prints the captured usage payload per picture. The corresponding documentation is the "Capturing API usage metadata" section in docs/usage/enrichments.md.

To get this working, make sure:

• Your Docling version includes the changes from PR #3632 (check your installed version and upgrade if needed).
• You haven't explicitly set usage_response_key=None.
• Your API's response contains a top-level usage object (which you confirmed it does when called directly).

Once on the correct version, the usage won't appear in the plain meta.description JSON — you need to call get_custom_part() to access it.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[gyanu2507]

Usage is captured, but it is not on the meta.description object you dumped. PR #3632 stores the provider payload as custom metadata:

usage = picture.meta.description.get_custom_part()["docling__usage"]

A JSON print that only shows confidence, created_by, and text will look empty even when capture worked. created_by: "not-implemented" is a separate annotation-author field. It is not a token counter and does not mean usage capture failed.

Checklist:

1. Upgrade to a Docling build that includes feat: capture picture description API usage #3632.
2. Leave PictureDescriptionApiOptions.usage_response_key at "usage" for an OpenAI-shaped API (or set a dotted path if your relay nests it).
3. Do not set usage_response_key=None.
4. Read get_custom_part(), not the public description dict.

Runnable example: docs/examples/picture_description_api_usage.py. Docs section: "Capturing API usage metadata" in docs/usage/enrichments.md.

If you are already on a post-#3632 release and get_custom_part() has no docling__usage key, the API response Docling saw did not contain that key. Dump the raw /chat/completions body from the same endpoint Docling calls (including any rewrite your gateway applies).

[nefayran]

Your install most likely predates the feature. Usage capture (#3632, merged 2026-06-22) first shipped in docling v2.105.0. Your meta dump has six keys (summary, description, classification, molecule, tabular_chart, code), and docling-core produces exactly those only from 2.71.0 to 2.76.0 (2026-03-30 to 2026-05-18). 2.77.0 added language and entities, 2.79.0 added keywords and topics. docling v2.104.0 already requires docling-core>=2.80.0 (pyproject.toml#L51), so you are on something older. Check with pip show docling docling-core, then pip install -U docling.

The replies above are right that the usage lands under docling__usage in the picture's custom fields. Once you are on a release that has it, it also shows up in a plain export_to_dict(). With docling-slim 2.134.0 against a local OpenAI-style server that returns usage:

{"created_by": "not-implemented", "text": "A cartoon duck.", "docling__usage": {"prompt_tokens": 415, "completion_tokens": 619, "total_tokens": 1034}}

If it is still missing after the upgrade, usage_response_key is None or names a key your gateway's response does not contain. The default is "usage" and dotted paths work (options).