Very slow for completions in long files with vscode-fauxpilot

[ravenscroftj]

The model can sometimes take several minutes to complete for long files. This was observed by @dabdine on mac but seems to be a general issue. I suspect that this is to do with the model having to factor in all the tokens in a really long prompt before making new predictions.

A possible technical trade-off could be to artificially limit prompt length to the last N words (this could be configurable). I imagine the model would still be able to make useful recommendations without the whole file as context.