Incorrect timestamp calculation in Qwen3VL Processor

[666zz666]

System Info

Should temporal_patch_size be used instead of merge_size for the timestamp calculation in this context?

transformers/src/transformers/models/qwen3_vl/processing_qwen3_vl.py

Line 155 in dfe3082

self.video_processor.merge_size,

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

None

Expected behavior

curr_timestamp = self._calculate_timestamps(
metadata.frames_indices,
metadata.fps,
self.video_processor.temporal_patch_size,
)

[zucchini-nlp]

Since both, merge and temporal sizes are 2, we don't see any difference in inference. I am not sure what is the intention behind though so let's tag Qwen team who contributed the model (@JJJYmmm).

transformers/src/transformers/models/qwen3_vl/processing_qwen3_vl.py

Lines 261 to 265 in dfe3082

	# @JJJYmmm frames are merged by self.merge_size, \
	# so we need to average the timestamps between the first/last frame within the temporal patch
	timestamps = [
	(timestamps[i] + timestamps[i + merge_size - 1]) / 2 for i in range(0, len(timestamps), merge_size)
	]

[JJJYmmm]

Thanks for the ping! 🤗 Currently, the video processor sets merge_size and temporal_patch_size to the same value.

Specifically, temporal_patch_size determines the temporal receptive field of the vision 3dconv, while merge_size controls the temporal compression ratio (initially, it's called temporal_merge_size). We may assign them different values in the future.

https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct/blob/main/video_preprocessor_config.json#L7-L8

[666zz666]

Thanks for your reply!
It would be beneficial to introduce a separate temporal_merge_size parameter. When using this Processor, I encounter different compression ratios for the spatial and temporal dimensions, which leads to a length mismatch between the video frames and the timestamps.
In the current codebase, merge_size effectively acts as spatial_merge_size. Since temporal_patch_size matches the temporal_stride of the Vision Encoder's first Conv layer, it essentially serves the purpose of temporal_merge_size.

[JJJYmmm]

I encounter different compression ratios for the spatial and temporal dimensions, which leads to a length mismatch between the video frames and the timestamps.

Sry, could you clarify why spatial compression matters here? The frame count and timestamps depend only on temporal_patch_size or video_processor.merge_size, which are currently equal. Spatial compression affects frame resolution only.