Bump jinja2 from 3.1.3 to 3.1.4 #17
Commits on Jul 18, 2024
-
-
-
-
-
llama : wrap llama_new_context_with_model in try/catch
This fixes a crash where ggml_vk_allocate fails in llama_kv_cache_init, but the exception is never caught.
-
kompute : fix ggml_vk_allocate failure control flow
The correct way to indicate an OOM condition is for alloc_buffer to return NULL. This fixes undefined behavior caused by passing an exception over the C boundary. The rest of the changes help fix VRAM leaks in GPT4All when model loading fails on GPU.
-
kompute : disable GPU offload for Mixtral
We haven't implemented the necessary GPU kernels yet. Fixes this crash: ggml_vk_graph_compute: error: unsupported op 'ARGSORT' GGML_ASSERT: /home/jared/src/forks/gpt4all/gpt4all-backend/llama.cpp-mainline/ggml-kompute.cpp:1508: !"unsupported op"
-
-
kompute : enable GPU support for 10 more model architectures
These are Baichuan, Bert and Nomic Bert, CodeShell, GPT-2, InternLM, MiniCPM, Orion, Qwen, and StarCoder.
-
-
-
-
-
vulkan : guard against multiple initialization
This trades a late heap-use-after-free for an early abort, which feels more correct.
-
-
-
-
-
-
llama : use the correct buffer type when we choose not to load on GPU
This fixes a regression in commit b2db03a ("llama: replace ngl=0 hack with llama_model_using_gpu").
-
-
-
kompute : fix dangling references in ggml_vk_graph_kompute
ggml_vk_get_tensor_aligned() returns a shared_ptr, not a reference, so we must copy the value.
-
kompute : avoid freeing device/instance until absolutely necessary
Eagerly freeing the instance when we are done with it is simple, but incurs an overhead, and more importantly, causes test-backend-ops crashes on the current proprietary NVIDIA driver. Instead, we now only cleanup device resources without freeing the device unless we actually need to change devices. And even when we free the device, we do not free the instance. We only free the instance when both the backend and all buffers have been unreferenced.
-
kompute : update ggml_vk_supports_op to fix false pos/neg
test-backend-ops hit assertion failures in ggml_vk_graph_compute because of ops we do not yet support. Some of the checks have to be made more restrictive because of features that were added to llama.cpp. We also claimed to not support no-op operations on certain data types, even though they are actually supported on all data types. There are now 243 passsing tests, instead of 150 without the fixes for false negatives. This also fixes complaints during LLM inference about unsupported NONE operations for the output tensor.
-
-
Bump jinja2 from 3.1.3 to 3.1.4
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]>
