Feature: Ollama integration

[jaroslaw-weber]

Feature request

I think it would be nice to have option to run this with local model through ollama.
I could implement it but seems there is no activity in the pull requests recently and worried my PR will get rejected. Please let me know if you greenlight this feature.

Why?

Gpt is paid, but would be nice to have free option.

Alternatives

No response

Additional context

No response

[MikeBirdTech]

This would be a massively beneficial feature. I would gladly offer my help with implementing it

@Nutlope

[mtompkins]

Echoing the support here. This would have a material impact on uptake as well.

[AntouanK]

If it's interesting, I have used it locally with LM Studio ( it supports any open source LLM )
https://lmstudio.ai/

--> caveat, most open-source have short context window ( 32k at best ) so large diffs won't work

the process I used :

• install / run LM studio ( get the model of your choice, and run the local server )
• find where the aicommits binary is ( it's a js script basically ). You can run which aicommits to find the path
• edit that file and replace the api.openai.com with localhost

• replace import Tn from "https"; with import Tn from "http";
• replace port 443 with 1234 ( or whatever port you set in LM Studio )
  

that should be it.

You can set any random openai key just to get it going ( for example aicommits config set OPENAI_KEY=sk-111 )

It works for me, but for small commits.

I hope it gets proper support with flags. And hopefully we'll get bigger context window on open-source LLMs soon.

Enjoy.

[newptcai]

I also made this work. Note that aicommits actually links to aicommits/dist/cli.mjs. If you do not want to overwrite the original cli.mjs, you need to make a copy of it and put it at the same location, e.g., aicommits/dist/cli-local.mjs. Then can create a symbolic link to it and call it something like aicommits-local. That gives you the option to use both open-ai and you local llm.

[newptcai]

I have to say, GPT3.5 works a lot better than local LLM.

[firstnapat]

If it's interesting, I have used it locally with LM Studio ( it supports any open source LLM ) https://lmstudio.ai/

--> caveat, most open-source have short context window ( 32k at best ) so large diffs won't work

the process I used :

• install / run LM studio ( get the model of your choice, and run the local server )
• find where the aicommits binary is ( it's a js script basically ). You can run which aicommits to find the path
• edit that file and replace the api.openai.com with localhost

• replace import Tn from "https"; with import Tn from "http";
• replace port 443 with 1234 ( or whatever port you set in LM Studio )
  

that should be it.

You can set any random openai key just to get it going ( for example aicommits config set OPENAI_KEY=sk-111 )

It works for me, but for small commits.

I hope it gets proper support with flags. And hopefully we'll get bigger context window on open-source LLMs soon.

Enjoy.

This work for me. Thank a lot 👍🏼👍🏼