perf: Allow all processed commits to be cached in the CompletionTimeQueryViewV2

[the-other-tim-brown]

Describe the issue this Pull Request addresses

The CompletionTimeQueryViewV2 will load the completion times incrementally and focus on loading the minimal set of files from the archived timeline. The issue is that it will potentially load the same file multiple times since it reads all entries from the file but only processes the instants in the provided range.

The inefficiency is seen when loading the view for instant at time T2 then at time T1. The LSMTimeline is read up to time T1 but only instants up to T2 are processed. The next request for T1 will re-read the file.

Summary and Changelog

• Update the filtering to only apply at the file level
• Update the ArchivedTimelineLoader interface to return the earliest instant time processed.
• Update the cursor to use the returned instant time

Impact

Makes loading the FileSystemView significantly more performant when there are archived instants

Risk Level

Low

Documentation Update

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[hudi-bot]

CI report:

• 8bd143f Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[danny0405]

+1, nice contribution!