JR's Garage

2026-08-02 Initial indexing of my vault on Mac mini (by Claude)

3 min read

// Implemented on My 24GB Mac mini, no external Thunderbolt or USB4 SSD // executed on a archived copy of my live vault

This file covers environment setup, model selection, and the indexing pipeline implementation.

Environment Setup

  1. Installed Homebrew, since it is the simplest way to install command line tools on macOS.
  2. Install Ollama using Homebrew, gives a local model runtime with an API endpoint at localhost port 11434. // looks like Ollama can launch frontier models
  3. Install Python 3.11 for orchestration scripts, embedding libraries, and vector database tools assume a recent Python version.
  4. Create a dedicated copy of the vault to test in

Download Models

  1. Pull the embedding model first, since it is small and will be used immediately for testing. // pulled via ollama pull nomic-embed-text. Default pulls into ~/.ollama/models. Tested and working
  2. Pull Qwen3 14B as your primary generation model. // pulled via ollama pull qwen3:14b. Testing via: (echo "Read this note and return 3-5 topic tags as a comma-separated list, nothing else:"; cat "/Users/josephrenner/Projects/garage/vault/Identify files by origin then type.md") | ollama run qwen3:14b

Vector Database and Embedding Pipeline

  1. Install a lightweight vector database library in Python, Chroma, run embedded directly in your script without a separate server process. // installed via pip3 install chromadb. Having issues with python finding to install. Created virtual environment instead and installed there
  2. Write a small indexing script that reads a markdown file, splits it into reasonably sized chunks, sends each chunk to the embedding model, and stores the resulting vectors with a reference back to the source file. Implemented as scripts/index_file.py:
  3. // Testing small scripts:
    1. // ls "Identify files by origin then type.md"
    2. // python3 scripts/index_file.py "Identify files by origin then type.md" --vault-root "/Users/josephrenner/Projects/vault copy for agent testing"
    3. // python3 scripts/index_file.py "tobiqmd mini cli search engine for your docs, knowledge bases, meeting notes, whatever. Tracking current sota approaches while being all local (by tobi).md" --vault-root "/Users/josephrenner/Projects/vault copy for agent testing"
  4. Tested this script against a small subset of files with garage in the name via scripts/index_subset.py, which finds vault-root .md files matching a name substring, reuses index_file() for each one, and prints a summary of files processed, failures, total chunks, and elapsed time.
  5. Write a basic query script that takes a test question, embeds it the same way, and returns the most similar chunks from your vector database. // created scripts/query.py
    1. // test question with python3 scripts/query.py "What does the term garage mean to me?" - returned 5 rational chunks
  6. Run the indexing script against your full archived vault copy. Implemented as scripts/index_all.py, which recursively walks the vault for markdown files, reuses index_file() for each one, and cleans up stale collection entries for files that were renamed or deleted, by diffing the current on-disk file list against source metadata already in the collection.
    1. // Ran script via python3 scripts/index_all.py --vault-root "/Users/josephrenner/Projects/vault copy for agent testing"
    2. // Test rename on 2026-07-17 093208 (from Whisper) to 2026-07-17 093208 (from Whisper) rename test - worked and was reported in the terminal
    3. See 2026-08-02 Testing vault index chunking and topic clustering (by Claude) for the chunking-bug and Ollama memory pressure investigation behind getting this to run cleanly across the full vault.

Follow on tasks

Graph View

Backlinks