Unload one or all Ollama models mid-workflow to free VRAM before image sampling. Wire your prompt through the passthrough socket so the node runs before the sampler. Node name: Ollama Unload (free VRAM). Located under Add Node > Ollama.
https://github.com/bradsec/ComfyUI_OllamaUnload
git clone https://github.com/bradsec/ComfyUI_OllamaUnload
comfy node install comfyui_ollamaunload