SigLIP2 discrepancy between HF implementation and original JAX implementation

[nmilosev]

System Info

Google Colab

• transformers version: 4.57.6
• Platform: Linux-6.6.105+-x86_64-with-glibc2.35
• Python version: 3.12.12
• Huggingface_hub version: 0.36.0
• Safetensors version: 0.7.0
• Accelerate version: 1.12.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.9.0+cu126 (CUDA)
• Tensorflow version (GPU?): 2.19.1 (False)
• Flax version (CPU?/GPU?/TPU?): 0.11.2 (gpu)
• Jax version: 0.7.2
• JaxLib version: 0.7.2
• Using distributed or parallel set-up in script?:
• Using GPU in script?:
• GPU type: Tesla T4

Who can help?

@yonigozlan @molbap @zucchini-nlp

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Steps to reproduce:

1. The issue is apparent when comparing official JAX implementation from @mitscha link to example with official SigLIP2 example from HF docs
2. When running inference in same model config the learned temperature and bias are different leading to "not ideal" performance in zero shot image classification. Please note that as of recent times the model is "operational", in previous version of HF transformers the model always gave 0% probs to all candidate labels.

Implementation	Temperature (Logit scale)	Bias
JAX Official	109.9	-15.9
HF Transformers 4.57.6	4.6994	-15.9324

Expected behavior

I expected learned temperature and bias to be similar.

When setting temperature to 100+ in HF implementation, model performance improves.

Please see attached notebook for a full example to replicate this issue.

SigLIP2_temperature_bias.ipynb

[vaibhav-research]

@zucchini-nlp if that’s okay, I can take a stab at this. looks interesting.

[vaibhav-research]

I ran a minimal forward-semantics repro and confirmed HF is behaving as expected.
• HF stores logit_scale in log-space: logit_scale=4.6994, so exp(logit_scale)=109.88, which matches the Big-Vision linear temperature (~109.9).
• HF logits match dot(normalized_embeds) * exp(logit_scale) + logit_bias (max error ~1e-6), and do not match the linear form * logit_scale.
• Therefore the reported temperature mismatch appears to be an apples-to-oranges comparison (log vs linear).

If there’s still a performance gap vs Big-Vision, the next step is to compare e2e logits on identical images/prompts with preprocessing aligned, the parameter values/forward scaling are consistent.
please let me know if my thought process is right.

cc @zucchini-nlp

[zucchini-nlp]

Thanks for investigating! Yeah, if the mismatch persists, we can try with smaller samples to compare. The model was tested for alignment though when adding it, so imo there should be no discrepancies with identical inputs