Skip to content

Spider2lite local272 has incorrect exec result #169

@ritahuja

Description

@ritahuja

local272 asks: For order 423, identify the product IDs, aisles, and positions from which to pick the exact quantities needed for each order line, ensuring that the total picked quantity for each product matches the cumulative quantities ordered without exceeding the available inventory in warehouse 1. Calculate the quantities to be picked from each location by prioritizing inventory with earlier purchased dates and smaller quantities, and ensure that picking respects the sequence and cumulative quantities of the order lines for products with multiple entries.

Gold answer is here: https://github.com/xlang-ai/Spider2/blob/main/spider2-lite/evaluation_suite/gold/exec_result/local272_a.csv

product_id,aisle,position,QUANTITY_TO_PICK
6520,A,16,22.0
4280,B,3,30.0
4280,C,20,30.0
6520,D,9,18.0

From my understanding of the question it should only pick from warehouse 1, but the golden answer shows:

4280.0 B 3.0 30.0

which is only available in warehouse 2:

PRODUCT_ID,QTY,WAREHOUSE,AISLE,POSITION
4280,37.0,1,A,4
4280,36.0,1,C,1
4280,39.0,1,D,18
4280,35.0,2,B,3
4280,19.0,2,C,7
4280,34.0,2,C,20

Suggestion:
Either reduce ambiguity from the question or accept alternative interpretations for gold answers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions