UX 개발 / AI 이미지 생성

Local Image and Video Agents on a Mac mini: A GUI-First Guide

Mac miniApple SiliconComfyUIOpen WebUIOllama로컬 AI

A compact desktop computer connected to prompt, workflow, image, and video cards

Set up ComfyUI, Ollama, and Open WebUI on an Apple Silicon Mac mini, generate images in chat, and prepare a separate workflow for short local videos. This guide targets M-series machines, including the M4, and prioritizes a graphical interface. You will enter a few installation commands, but you will not write a Python agent or build a multi-agent framework.

1. Choose a stack that fits the Mac

Start with ComfyUI + Open WebUI, adding Ollama when you want the conversation model to run locally too. This is a recommendation for this tutorial, not a claim that one stack is universally the easiest or most popular.

ComponentIts jobWhere it runs
ComfyUI / Comfy DesktopLoad image/video models and execute workflowsYour Mac, using the supported Apple GPU backend
Open WebUIProvide the chat interface and image-generation integrationYour Mac
OllamaRun the language model that discusses ideas and calls toolsYour Mac with a downloaded local model
Gemini API — optionalReplace the local conversation modelGoogle’s servers; rendering can remain local

The image path is: your message → conversation model → Open WebUI image tool → ComfyUI workflow → image in chat. The chat model decides what to request; the image model renders pixels. A tool-calling assistant is enough for this first agent.

The original Forge proposal needs several corrections:

  • SD WebUI Forge is a real option, but CUDA-oriented instructions and memory optimizations cannot be carried over unchanged to Apple Silicon. The Apple GPU backend in PyTorch is MPS, not CUDA. ComfyUI has a documented Apple Silicon desktop installation path. (Apple’s MPS guide, Comfy Desktop for macOS)
  • AUTOMATIC1111-compatible /sdapi/v1/txt2img is an image endpoint. It does not provide a general video API, and support for an extension does not make all FLUX or Wan workflows interchangeable.
  • Quantization formats, negative prompts, CFG, and sampling steps depend on the model and backend. FP8 is not a universal Mac optimization.
  • Five-minute installation estimates and blanket reliability rankings omit model downloads, hardware differences, and tool-calling failures.

2. Check your Mac and set a small first goal

Open Apple menu → About This Mac and note the chip, memory, and macOS version. This article assumes Apple Silicon, not an Intel Mac mini. The current Ollama macOS documentation requires macOS 14 or later; Comfy Desktop documents macOS 13 or later. The Homebrew-based commands below assume macOS 15 or later, following Homebrew’s current supported-platform policy. (Ollama requirements, Comfy Desktop requirements, Homebrew support)

Unified memorySuggested starting scope, not a compatibility guarantee
8 GBA small image workflow by itself; combining a local LLM and renderer is restrictive
16 GBSD 1.5 at 512 × 512, batch size 1, plus a small chat model with careful unloading
24–32 GBMore room for image experiments; test short video workflows individually
48–64 GB or moreMore capacity for video experiments; speed and MPS support still need checking

These are conservative planning suggestions, not measured benchmarks. Unified memory is shared by macOS, applications, and the GPU. A model’s download size is not its peak memory use, and an NVIDIA “8 GB VRAM” claim does not promise success on an 8 GB Mac.

Plan roughly 20 GB of free storage for the initial image setup, and additional tens of GB for video models and outputs. Exact requirements depend on the selected files. Keep Activity Monitor’s Memory Pressure visible while testing.

3. Install Comfy Desktop locally

Download the Apple Silicon installer from the official Comfy download page, open the .dmg, and drag the app into Applications. Launch it and choose Local in the current onboarding flow. Earlier releases may call this New Install. Choose the recommended stable local installation and let the app prepare Python and dependencies. Choose an SSD location with room for models. (Desktop onboarding)

Comfy Desktop onboarding with Local selected and an optional Express Install setting

Official ComfyUI documentation image: choose Local to install the renderer on this computer. (Source: Comfy Desktop overview, checked 2026-10-25)

Give the local instance a recognizable name. From its ⋮ → Manage → Startup Args, set an explicit address and port for this tutorial:

--listen 127.0.0.1 --port 8188

Restart the instance and open http://127.0.0.1:8188 in your browser. If the app reports a port conflict or chooses a different port, use the actual address shown by its startup logs throughout this guide. The instance’s Manage panel and startup flags are documented separately. (Manage installations, network flags)

Both services will run natively on the same Mac, so loopback is sufficient. You do not need --listen 0.0.0.0, port forwarding, or Docker networking for this setup.

4. Generate one image in ComfyUI first

Use Stable Diffusion 1.5 for the first connectivity test. It is an older, smaller baseline; this choice is about getting the workflow working before testing larger models.

  1. Open the official text-to-image tutorial. Download its workflow PNG and drag that file into ComfyUI; the PNG contains the workflow metadata.
  2. Download the SD 1.5 FP16 checkpoint linked by ComfyUI. Review its model card and license.
  3. Put it in the active instance’s model library under models/checkpoints/. Use Manage → Storage to locate configured model directories; a shared Desktop library may be outside the app installation.
  4. Refresh the model list or restart the instance. In Load Checkpoint, select the exact downloaded filename.
  5. Set Empty Latent Image to 512 × 512, batch size 1. Set KSampler to 20 steps, CFG 7, sampler euler, scheduler normal, denoise 1.
  6. Enter the prompts below, click Run/Queue, and confirm an image appears in Save Image. Locate its saved output through the instance’s configured output directory.
Positive: blue ceramic teapot, wooden table, soft window light,
product photograph, simple background

Negative: text, watermark, blurry

The A–F diagram shows the checkpoint, latent canvas, positive/negative text, sampler, decoder, and saver. It is an official example, not a result generated on the reader’s Mac.

SD 1.5 workflow with the checkpoint, latent image, two text encoders, sampler, VAE decoder, and Save Image connected

Official ComfyUI workflow explanation. Read the labels at full size. (Source: text-to-image tutorial, checked 2026-10-25)

Do not add the chat layer until this standalone render works. Larger image models can come later; the site’s ComfyUI and FLUX introduction explains the separate model-file layout.

5. Install Ollama for local conversation

Get the macOS app from the official Ollama download page, install it, and launch it. Allow its CLI setup if prompted. In Terminal, download a small model with tool support:

ollama pull qwen3:4b

Qwen3’s official Ollama listing includes tool support and the 4b variant. This is a modest starting model, not a guarantee of reliable tool use. Use a larger model only after checking memory headroom.

To reduce memory retained between requests, set Ollama’s default keep-alive to zero, then fully quit and reopen the Ollama app:

launchctl setenv OLLAMA_KEEP_ALIVE "0"

This makes subsequent responses release the model sooner, at the cost of reloading it for the next turn. A client-supplied keep_alive overrides this default. Check ollama ps during rendering; if the model stays loaded, check the client’s setting or use Open WebUI’s Eject control when idle. The environment setting may need reapplying after signing out or restarting macOS. (Ollama configuration and unloading, Open WebUI’s unload control)

6. Install Open WebUI without a custom script

This route uses Open WebUI’s supported native Python installation so both services can use 127.0.0.1. If Homebrew is not installed, follow the installer and shell setup on the official Homebrew website first.

Open a new Terminal window and run:

brew install python@3.12
mkdir -p ~/AI/open-webui
cd ~/AI/open-webui
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install open-webui
DATA_DIR="$PWD/data" open-webui serve --host 127.0.0.1 --port 8080

The commands install the application in its own environment; they do not create an agent program. Leave this terminal running. Open http://127.0.0.1:8080 and complete the first administrator account setup. Keep starting from this same directory so the data and session-secret location remain consistent. The current Open WebUI quick start supports Python 3.11/3.12 and documents the native launch command.

Under Settings → Admin → Connections, check the Ollama connection at http://127.0.0.1:11434. Select qwen3:4b in a chat and ask a simple text question. Confirm conversation works before configuring images. If the connection was not detected automatically, add it there. (Ollama connection guide)

7. Connect the working image workflow

In ComfyUI, enable Dev Mode in Settings if necessary, then choose File → Export (API). Save the already tested SD 1.5 workflow as sd15-api.json. Ordinary workflow JSON with a top-level nodes array is a different format. API exports use node IDs as top-level keys. (ComfyUI API example)

In Open WebUI, open Settings → Admin → Images, enable Image Generation, and configure Create Image:

SettingValue for this tutorial
EngineComfyUI
Base URLhttp://127.0.0.1:8188 — match the running instance
ModelThe exact SD 1.5 checkpoint filename
Image size / steps512x512 / 20
WorkflowUpload sd15-api.json
API keyEmpty for the local instance unless you configured authentication

Refresh/test the connection and save. The official integration page is a community-contributed tutorial hosted in Open WebUI’s documentation. Its Docker address examples differ from this native setup. (ComfyUI integration)

Open WebUI image-generation settings with the ComfyUI engine, server address, workflow upload, and node mappings

Official documentation screenshot from Open WebUI. Use the loopback URL above for this guide; screenshot values are examples. Full size. (Source: ComfyUI integration, checked 2026-10-25)

Map controls to the correct nodes

Read your exported JSON in a text editor. Find the following class types and input keys, then enter their actual IDs in Open WebUI. The example IDs below belong to the standard basic workflow and can change after editing or importing.

Open WebUI fieldKeyNode to identifyTypical example ID
PrompttextCLIPTextEncode connected to KSampler’s positive input6
Modelckpt_nameCheckpointLoaderSimple4
WidthwidthEmptyLatentImage5
HeightheightEmptyLatentImage5
StepsstepsKSampler3
SeedseedKSampler3

Leave the negative prompt in the exported workflow; do not also map the positive prompt to the negative text node. Keep the remaining sampler settings in that file. A different model family may use unet_name or another input instead of ckpt_name. (Integration mapping reference)

Open WebUI node mapping fields pairing each input key with the workflow's node IDs

Official Open WebUI documentation example. It shows unet_name for a different workflow; use ckpt_name for this SD 1.5 checkpoint loader. (Source: node mapping, checked 2026-10-25)

8. Generate your first image in chat

Keep qwen3:4b selected as the chat model. In Workspace → Models → Edit, ensure Image Generation, Builtin Tools, and the image-generation tool category are enabled. Your account must also have image-generation permission. In the message input’s Integrations menu, switch Image on. (Chat image requirements)

With Function Calling set to Native, try:

Generate one image of a blue ceramic teapot on a wooden table.
Use soft window light and a simple background.
Call the image-generation tool now; do not only describe the image.

Watch ComfyUI’s queue. Success means the queue runs, Save Image receives an output, and that image appears in the Open WebUI conversation. The local LLM may reload after rendering to write a final response.

If the model only replies with text, temporarily select Legacy under Chat Controls → Advanced Params → Function Calling. With Image on, Legacy routes each submitted message to the image backend; switch Image off for ordinary conversation. This helps distinguish tool-selection problems from a broken renderer. It is a different interaction mode, not an autonomous tool decision. Native tool calls depend on the language model’s ability. (Native and Legacy behavior)

Start with a short context. If needed, set Ollama’s num_ctx to 8192 in the model/chat advanced settings, while watching memory; very small contexts can truncate tool definitions. (Context configuration)

9. Add short video as a separate ComfyUI experiment

Open WebUI’s built-in image integration is not a general video-agent interface. Keep the working image configuration. For a first local video, use ComfyUI directly; attaching video generation to chat would require a separate tool/integration beyond this GUI walkthrough.

Start with the older, smaller Wan2.1 T2V 1.3B example rather than assuming a 14B model or a new release will fit. It generates video from text, not from your teapot image. Model/backend compatibility and practical speed still need testing on your particular Mac. (Wan project, ComfyUI’s native Wan guide)

  1. Stop other generation jobs and unload Ollama using Eject or ollama stop qwen3:4b.
  2. In ComfyUI’s Template Library, open Wan 2.1 Text to Video, or download the workflow linked by the official guide.
  3. From the Comfy-Org model repository, download the following files into the active model directories:
models/
├── diffusion_models/wan2.1_t2v_1.3B_fp16.safetensors
├── text_encoders/umt5_xxl_fp16.safetensors
└── vae/wan_2.1_vae.safetensors
  1. Select those files in Load Diffusion Model, Load CLIP, and Load VAE. Set the CLIP type to wan; choose device: cpu in Load CLIP’s advanced options if needed to keep text encoding off MPS. The FP16 encoder is large, and CPU placement still uses unified memory. The CLIPLoader implementation exposes this CPU option. The template may initially select an FP8 encoder; change it explicitly for this conservative first test.
  2. For a compatibility test, set the latent video’s size to 512 × 320, length 17, batch size 1. Keep the template’s sampler settings initially. Use 8 FPS in Create Video for a clip of roughly two seconds. This small test is not a quality benchmark; increase resolution and frames only after success.
  3. Try: A blue ceramic teapot on a table, a slow gentle camera push-in, soft daylight, stable composition. Run one job.
  4. In the current template, check Create Video → Save Video. Choose MP4/H.264 if available and find the file under the configured output directory. Older templates may end with SaveAnimatedWEBP and save an animated WebP instead. (Current workflow JSON, Save Video implementation)

Official Wan2.1 workflow identifying diffusion model, text encoder, VAE, video dimensions, prompts, sampler, and animated output

Official ComfyUI reference image from an earlier workflow. Its BF16/FP8 selections and WebP saver differ from the FP16 files and current video saver described above. Full size. (Source: Wan2.1 guide, checked 2026-10-25)

If it runs out of memory or hits an unsupported MPS operation, stop at that failure and record the model, ComfyUI/PyTorch versions, and error. Do not treat this as a promise of video generation on every 16 GB Mac, force an FP8 dtype, or disable memory safeguards. Use a smaller/compatible workflow or move rendering to another machine if the requirement exceeds yours.

For image-to-video, Wan2.2 TI2V 5B’s official example is a separate later experiment: it uses different diffusion/VAE files and a Load Image branch. Its NVIDIA offloading memory guidance is not an Apple Silicon guarantee. Do not replace the 1.3B checkpoint and expect the same graph to work.

10. Use Gemini only if you want a cloud conversation model

For this optional hybrid path, create an API key in Google AI Studio. In Open WebUI’s Admin → Connections, add an OpenAI-compatible connection with Google’s official base URL:

https://generativelanguage.googleapis.com/v1beta/openai/

Add the Gemini API key, refresh the available models, and select a currently available model with tool support. Keep Images → Engine = ComfyUI. Configuring a Gemini image engine would use Google’s image service instead. Conversation data sent to Gemini leaves your Mac; this route has local rendering and cloud reasoning, with the provider’s usage terms and costs. (Google’s compatibility instructions)

11. Troubleshoot one layer at a time

SymptomFirst check
ComfyUI alone failsModel files, active model paths, MPS error, and Memory Pressure
Open WebUI cannot connectComfyUI’s actual address/port; both processes must be running
Workflow rejectedAPI export rather than ordinary editor JSON; valid checkpoint filename and node mappings
A response contains only textImage toggle, model capabilities, permissions, then Native tool behavior; test Legacy
Image renders but chat cannot display itBase URL must match the backend origin; check the image-fetch error before changing security settings
Mac swaps heavilyBatch size 1, smaller canvas, shorter context, unload other models
Video model reports dtype/operation errorsVerify that model and precision against the MPS backend; CUDA flags do not fix Metal compatibility

Open WebUI’s image troubleshooting guide covers workflow validation and image-fetch failures. If you later choose Docker Desktop, container 127.0.0.1 refers to the container; host addressing and backend reachability must be configured separately. Do not mix those URLs into this native installation.

12. Restart, save, and expand gradually

For the next session, launch Ollama, start the local ComfyUI instance, and reopen Open WebUI with:

cd ~/AI/open-webui
source .venv/bin/activate
DATA_DIR="$PWD/data" open-webui serve --host 127.0.0.1 --port 8080

Keep the working workflow JSON, API export, checkpoint filename, seed, prompts, and app versions. Back up Open WebUI’s data directory and the separately configured ComfyUI models, workflows, and outputs. Stop Open WebUI with Ctrl+C before copying its database directory.

The first useful milestone is one image generated through chat and saved locally. After that, change one variable at a time: a better image model, a stronger conversation model, or a short video workflow. The installation and interface details here were checked against official sources on 2026-10-25; this article does not claim measured generation times or a completed inference run on a specific Mac mini configuration.

Related