clams.backends package

Optional model-backend helpers for CLAMS apps.

Each backend is a separate submodule. Heavy dependencies (e.g., torch, transformers) are NOT pulled in by the base clams-python install; users opt in via pip extras such as pip install clams-python[hf] for the HuggingFace transformers backend.

Submodules

clams.backends.hf

HuggingFace transformers backend helpers.

Two general loaders that wrap the device / kwargs / inference-mode boilerplate every HF-backed CLAMS app does identically:

  • load_hf_model() – from_pretrained() flow for any model class (instruction-tuned LLMs/VLMs, encoder-only classifiers, vision/audio feature extractors, etc.). Use when the app needs raw access to the underlying model and processor.

  • load_hf_pipeline() – task-level transformers.pipeline() flow (ASR, NER, text classification, zero-shot, etc.). Use when pipeline-level inference is sufficient.

torch and transformers are optional dependencies. Install them via the [hf] extra:

pip install clams-python[hf]

Imports are lazy: this module can be referenced from clams.app without triggering an ImportError on a base clams-python install. The ImportError only fires when a loader is actually called without the extras.

clams.backends.hf.load_hf_model(model_id: str, model_cls, processor_cls=None, dtype=None, device: str | None = None, padding_side: str | None = None, revision: str | None = None, model_kwargs: dict | None = None, processor_kwargs: dict | None = None, move_to_device: bool = True) → Tuple[Any, Any, str][source]

Load a HuggingFace transformers model via from_pretrained and return it ready for inference.

Parameters:

  • model_id – HuggingFace model identifier (e.g., a Hub repo name or a local path) forwarded to from_pretrained.

  • model_cls – a transformers model class (e.g., AutoModelForCausalLM, AutoModelForImageTextToText, ConvNextV2Model, ViTModel, …). Whatever supports from_pretrained().

  • processor_cls – a processor / tokenizer / feature-extractor class with from_pretrained(). Defaults to transformers.AutoProcessor. Pass transformers.AutoTokenizer, transformers.AutoImageProcessor, etc. for narrower cases. Pass None explicitly to skip processor loading entirely (the returned processor in that case is None).

  • dtype – torch dtype for the model (e.g., torch.bfloat16). When None (default), no torch_dtype kwarg is forwarded to from_pretrained – the model class uses its own default (typically float32). Set explicitly for low-precision LLM inference.

  • device – target device string (e.g., 'cuda', 'cpu', 'cuda:0'). When None (default), the helper auto-detects cuda availability and falls back to cpu.

  • padding_side – if set (typically 'left' for decoder-only models doing batched generation), the helper configures the underlying tokenizer’s padding_side and – when no pad token is set – uses the EOS token as the pad token. Leave None for encoder / non-batched cases (the tokenizer’s own default is preserved).

  • revision – optional Git revision (commit hash, branch name, or tag) on the Hub repository to pin the download to. When set, forwarded as revision=... to both model_cls.from_pretrained and processor_cls.from_pretrained, ensuring the model and processor are loaded from the same commit. Strongly recommended for production: pinning a commit hash makes the analyzer artifact reproducible and immune to upstream silent updates. Apps calling this helper directly should record the same hash on analyzer_version (or analyzer_versions) in metadata.py so the output MMIF identifies the exact artifact. Apps inheriting from ClamsHFPromptableApp do not call this helper – the base class reads analyzer_versions from the app metadata and forwards the resolved revision automatically.

  • model_kwargs – extra kwargs forwarded to model_cls.from_pretrained() (e.g., {'use_safetensors': True, 'add_pooling_layer': False}).

  • processor_kwargs – extra kwargs forwarded to processor_cls.from_pretrained() (e.g., {'use_safetensors': True, 'use_fast': True}).

  • move_to_device – when True (default), the helper moves the loaded model to the resolved device and switches it to eval() mode – the right behavior for a “ready for inference” app loader. When False, both steps are skipped; the model is returned in the state from_pretrained left it (on CPU, in train mode). Use False for library-style HF wrappers that defer device placement and inference-mode switching to a downstream consumer (e.g. an extractor class that may be combined with a head and only then placed on a device by the wrapping classifier). The returned device is still the resolved target, so the consumer can use it later for its own .to(device) call.

Returns:

(processor, model, device) tuple. processor is the loaded processor/tokenizer/feature-extractor (or None if processor_cls was explicitly set to None). device is the resolved device string (the model was moved there iff move_to_device=True).

Return type:

Tuple[Any, Any, str]

Raises:

ImportError – if torch or transformers is not installed. Install the [hf] extra to fix.

clams.backends.hf.load_hf_pipeline(task: str, model_id: str, device: str | int | None = None, revision: str | None = None, model_kwargs: dict | None = None, pipeline_kwargs: dict | None = None) → Tuple[Any, str | int][source]

Load a HuggingFace transformers.pipeline() for task and return it ready for inference. Wraps the device / revision / kwargs-forwarding boilerplate that every pipeline-backed CLAMS app does identically. Use this for apps wrapping a task-level pipeline (ASR via "automatic-speech-recognition", NER via "token-classification", text classification, zero-shot, etc.); use load_hf_model() instead when the app needs raw access to the underlying model / processor (e.g., for custom chat-template formatting or batched generate calls).

Parameters:

  • task – pipeline task string forwarded to transformers.pipeline() (e.g., "automatic-speech-recognition", "token-classification").

  • model_id – HuggingFace model identifier (Hub repo name or local path) forwarded to pipeline(model=...).

  • device – target device. Accepts the string form ('cuda', 'cpu', 'cuda:0') for parity with load_hf_model(), or the integer form accepted natively by pipeline (-1 for CPU, 0+ for GPU index). When None (default), auto-detects cuda availability and falls back to cpu (string form).

  • revision – optional Git revision (commit hash, branch, or tag) on the Hub to pin the download to. Strongly recommended for production; see load_hf_model() for rationale.

  • model_kwargs – extra kwargs forwarded to the underlying model.from_pretrained() via the pipeline(model_kwargs={...}) channel.

  • pipeline_kwargs – extra kwargs forwarded directly to transformers.pipeline() (e.g. generate_kwargs, tokenizer, feature_extractor, batch_size, framework). model, task, device, revision, and model_kwargs are owned by this helper – explicit helper args take precedence if any collide.

Returns:

(pipeline, device) tuple. device is the resolved device the pipeline is on, in the form it was passed (or the auto-resolved string form when device=None).

Return type:

Tuple[Any, Union[str, int]]

Raises:

ImportError – if torch or transformers is not installed. Install the [hf] extra to fix.