Embedded Codon JIT: how to interact with Geneator function

[yfinkelstein]

Hello,
I'm trying to expose a C/C+ side extern function as a generator for Codon-side functions as well as the opposite: invoke a generator function from C/C++ and iterate over its yields. It would be great to have some guidance on how it can be done the right/best way.
Thanks!

[arshajii]

I would suggest wrapping the generator and implementing an API like gen_next() / gen_done() that can be used to advance it & get the next value. While it might be possible to leverage C++ coroutines to interact with Codon generators directly (since Codon uses the same machinery under the hood to implement generators), that could get hairy due to potential ABI differences, GC, etc.

There's some reference material about generator methods here: https://docs.exaloop.io/libraries/api/internal/types/generator/. The important ones are:

• gen.__done__(): returns a bool indicating whether the generator is done
• gen.__resume__() resumes the generator
• gen.__promise__() returns a pointer to the generator "promise", i.e. the yielded value (so for Generator[int], this would be Ptr[int] or int64_t* in C)