Skip to content

[Prompt API] Restricting the Phi-4-mini-instruct model to a 9216 token context window makes no sense given the hardware required to run it. #1224

@moviescripter

Description

@moviescripter

In the current Prompt API implementation in the Chrome Dev and Edge Dev browser versions, the context window is restricted in both to a fixed 9216 tokens. I can understand why this is harmonized between browsers but Gemini Nano can run on much less capable hardware (just 16GB of RAM) whereas Phi-4-mini-instruct requires quite powerful hardware (5.5GB VRAM, essentially a mid-range gaming PC). So we're saying: ' Edge must run on very capable hardware but at the same time we're handicapping the Edge model so it's no better than the Chrome model'. In my specific use case I want Phi-4-mini to be able to analyze movie screenplays in Fountain format. Any feature screenplay will be at least 80 pages but 9216 tokens accommodates only 20 to perhaps 30 pages at best - a fairly short movie , yet Microsoft documents the incredibly useful 128K context window its model could offer.

Metadata

Metadata

Assignees

Labels

Prompt APIIssues about the Prompt API in Edge

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions