You might need to utilize the gpu_memory_limit and/or lora_on_cpu config choices to avoid jogging outside of memory. If you continue to operate away from CUDA memory, you are able to attempt to merge in program RAM https://www.imtoken.icu/