My Local AI Solution with Mac Mini
Hardware
The foundation of my local AI setup is the Apple Mac Mini Desktop Computer with M4 Pro chip, equipped with 24GB of RAM. This compact powerhouse provides the processing muscle needed for running AI models efficiently without relying on cloud services.
Storage Setup
In addition to the Mac Mini's internal storage, I use two 1TB external drives: one dedicated to photos (/Volumes/Photos) and another for videos (/Volumes/Videos). There's also a 2TB external drive (/Volumes/Backup) that serves as the primary backup drive via Time Machine. For the external drives, I recommend models like the BUFFALO External SSD 1TB (scaled up to 2TB for the backup).
To maintain a neat and organized desk setup, I connect the external drives using new, dedicated cables such as the Micro B to USB C, USB 3.0 Cable External Hard Drive Cord. This direct connection to the Mac Mini's USB-C ports eliminates clutter from mismatched or worn-out cables, resulting in a clean, professional look with optimal data transfer speeds.
As an alternative, if you prefer to reuse existing USB-A cables from your drives, you can employ a compact USB 3.0 Female to Thunderbolt 4/3 Adapter. This adapter plugs into the Mac Mini's Thunderbolt ports, allowing seamless compatibility while keeping the setup tidy—though the direct cable option is generally sleeker with fewer points of failure.
Backup Process
Backing up the /Volumes/Photos and /Volumes/Movies drives to the /Volumes/Backup drive is handled through macOS's built-in Time Machine feature, which provides automated, incremental backups. Here's the step-by-step process:
- Connect all external drives to your Mac Mini and ensure they are mounted (appearing as /Volumes/Photos, /Volumes/Movies, and /Volumes/Backup).
- Go to System Settings (or System Preferences on older macOS versions) > Time Machine.
- Select the /Volumes/Backup drive as your backup disk if it's not already configured.
- Click "Options..." (or "Select Disk..." if setting up fresh) and review the exclude list. By default, Time Machine backs up your internal drive, but external volumes like Photos and Movies may be excluded to save space. Remove them from the exclude list to include them in backups.
- Time Machine will now automatically back up the selected volumes hourly (for changes), daily, and weekly, retaining versions as space allows. You can trigger a manual backup by clicking the Time Machine menu bar icon and selecting "Back Up Now."
- For added reliability, monitor the backup progress and verify restores periodically by entering Time Machine (via the menu bar) and browsing the backed-up files.
This process ensures your photos and videos are safely duplicated without manual intervention, protecting against data loss.
AI Software
My local AI stack runs entirely on the Mac Mini, leveraging open-source tools for flexibility and privacy:
- Ollama: A lightweight framework for running large language models (LLMs) locally. It handles model downloading, inference, and API serving, making it easy to deploy AI without internet dependency.
- OpenWebUI: A user-friendly web interface built on top of Ollama, allowing browser-based interaction with models through chat interfaces, custom prompts, and session management.
- N8N: An automation platform for creating workflows. I use it to chain AI models with other tools, such as automating data processing, API calls, or integrating with local files for AI-driven tasks.
AI Models
I run the following models via Ollama, each tailored to specific use cases:
- llama3.1:8b: A compact 8-billion-parameter version of Meta's Llama 3.1 model, optimized for general text-based tasks like chatting, writing assistance, summarization, and reasoning. It's efficient for everyday queries on modest hardware.
- llama3.2-vision:11b: An 11-billion-parameter multimodal model from the Llama 3.2 series, incorporating vision capabilities. It excels at image analysis, such as describing photos, answering questions about visual content, or generating text based on images (e.g., captioning or object detection).
- x/flux2-klein:9b: A 9-billion-parameter text-to-image generation model (a smaller variant of the Flux series), designed for creating high-quality images from textual descriptions. It's useful for creative tasks like generating artwork, visualizations, or custom graphics.
- qwen2.5-coder:14B-instruct: A 14-billion-parameter instruction-tuned model from the Qwen 2.5 family, specialized in coding. It handles code generation, debugging, explaining programming concepts, and assisting with software development in languages like Python, JavaScript, and more.
Shopping List
Apple Mac Mini Desktop Computer with M4 Pro chip: Compact, high-performance machine with 24GB RAM ideal for local AI workloads.
Buy Now
Micro B to USB C, USB 3.0 Cable External Hard Drive Cord: High-speed cable for connecting external drives to USB-C ports.
Buy Now
USB 3.0 Female to Thunderbolt 4/3 Adapter: Compact adapter for bridging USB-A devices to Thunderbolt/USB-C ports.
Buy Now
BUFFALO External SSD 1TB: Portable, fast SSD for storing photos, videos, or backups (available in larger capacities like 2TB).
Buy Now