Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support docker install and Ollama #9

Open
chllei opened this issue Aug 2, 2024 · 1 comment
Open

Support docker install and Ollama #9

chllei opened this issue Aug 2, 2024 · 1 comment

Comments

@chllei
Copy link

chllei commented Aug 2, 2024

Thank you very much for developing such a successful and practical educational app.

I am particularly concerned about privacy. I would like to know if it is possible to use smaller models (for example 70B or even 8B) through Ollama as the inference backend on local Docker containers to set up this tutor?

@namanbnsl
Copy link

I think this might require a writeup of some part of the code because as far as I understand, the website is directly interacting with the Together API. The Together API is OpenAI compatible and so is Ollama. Maybe you can try to change API calls to the ollama openai endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants