My Second Brain. Finding conversations and documents.

Before AI can answer a question for me, I often spend time explaining what the question is about. The appointment is in my calendar, a change is in an email, my notes are on a reMarkable and the document is on my Mac.

I am building an assistant that can consult my own sources. I call it my Second Brain. I want to find my renovation records or prepare for a work meeting without first copying all the information into a chat myself.

My Mac is the workspace. OpenClaw provides the gateway, and my own MCP connection gives an AI client access to my search tools. I start below with the questions I ask. The architecture and code are in the technical section.

About the examples. The reading rounds took place on 4 October 2026; the technical account follows the version of 5 October. In that version, resuming earlier work, further context loops and parts of local model execution remain development work.

Explaining less from scratch

My test is practical: does this help me prepare for an appointment, recover an earlier decision or open a source I had forgotten? An answer is useful only when I can see what supports it.

Software for my own workspace

This project does not have to become a general product. I can shape it around my devices, history and the apps I want to keep using. A close integration with my Mac can be worthwhile for me even if I would never want to maintain it for other users.

I also enjoy the work. I build something for a recurring question, try it and discover what is missing. I am putting that time into an environment I want to use myself.

My renovation records and a work meeting

On 4 October 2026, I asked about my own house records and an upcoming meeting for Vanas. These examples show the sources found and the answers built from them. The address, names and business details are redacted. Times use Europe/Amsterdam.³³

What does my building history report say?

I wanted to know which parts of my house had heritage value and what to consider during renovation. Correspondence from 1986 appeared first. It helped establish building history but did not answer what I could change now.

Using files.local_search with bouwhistor, I then found a building history report from 2023. The search covered the document index, Spotlight and local filenames. Three file references did not necessarily mean three independent studies.³³

{
  "actionType": "files.local_search",
  "request": {"query": "bouwhistor", "limit": 6}
}

{
  "count": 3,
  "providersQueried": [
    "openclaw.ai_documents",
    "spotlight_local",
    "bounded_local_filename"
  ],
  "results": [{
    "displayName": "2023 06 06 Bouwhistorische verkenning [address removed].pdf",
    "source": "spotlight_local",
    "sizeBytes": 6172492,
    "indexed": true,
    "absolutePathExposed": false
  }]
}

I then searched within the retrieved content for valuation, interior and beam structures. Page 45 contained “Internal valuation”. The passages described mouldings, profiled beams, crawl-space arches and the marble hallway floor.³³³⁴

The report gives the historic building mass and the listed interior elements a high valuation. The partition with sliding doors and built-in cupboards receives a positive valuation. Those categories need to remain distinct when I compare my plans with the report.

This summarises report passages; it is not a new building inspection. The valuation also does not by itself define legal protection. The Dutch Cultural Heritage Agency explains that protection of a listed building can cover the entire immovable property, including its interior. Specific alterations and energy improvements call for consultation with the municipality.³¹³²³⁴

What this gave me was the relevant parts of my own report in one place. The assistant had to preserve the valuation categories and the study year. Reindexing a 2023 report did not turn it into a new study.

files.local_search
query: "bouwhistor"
count: 3
source: "spotlight_local"

knowledge.search
sectionTitle: "Page 45"
sourceKind: "icloud_drive"
sourceMode: "local_pdf"

When is my next meeting for Vanas?

My next question was: “When is my next meeting for Vanas? Give me a prep.” The assistant first needed to find a work appointment, then retrieve the background.

The calendar query covered 4–6 October. This read used local Apple Calendar data and the Outlook widget cache on my Mac. Reading those records did not use a network connection.³³

{
  "actionType": "calendar.range",
  "request": {
    "fromDate": "2026-10-04",
    "toDate": "2026-10-06",
    "limit": 10
  }
}

{
  "sourceAdapter": "microsoft-outlook-calendar-widget",
  "title": "[internal subject line removed]",
  "start": "2026-10-05T08:30:00+02:00",
  "end": "2026-10-05T09:00:00+02:00",
  "allDay": false,
  "location": "Microsoft Teams-vergadering"
}

The appointment found was on 5 October, 08:30–09:00, on Teams. Its subject gave the assistant a starting point for earlier correspondence through meeting.prep, mail.search and mail.get_message.³³

The email contained exactly the sort of trap I want to catch. It mentioned a Monday meeting but was received on 28 May 2026. “Monday” could not simply mean tomorrow. The calendar supplied the current appointment time; the email supplied older background.

Suggested preparation: check whether the old comparison still fits the current question, put the assumptions side by side and decide what needs to be resolved in the meeting.

That is a proposal based on the sources, not an instruction quoted from the appointment. I could retrieve the background without treating an old email as a current request.

Finding a guide again

The term monument also found groene_gids-binnengevelisolatie_van_monumenten.pdf through bounded_local_filename. That call located the file without reading its contents. Recovering the guide without remembering its folder was already useful to me.³³

What the AI receives for these questions

The model receives targeted tool results: fields, passages, source labels and dates. It does not automatically get my whole drive or screen. Finding a document can require a second call to read it.

These examples show the reading routes used. They do not establish that every question runs vector search, graph expansion and a neural reranker together. No emails were sent and no appointments changed as part of these questions.³³

Who is waiting for whom?

A read email can still be waiting for me. A question in one of my sent messages can be waiting for somebody else. I want to see the exchange before deciding who needs to reply.

A limited email selection

On 4 October I used the planner, then mail.search, for a small selection. The search returned four candidates and possiblyTruncated: true, so more email could fall outside the result.³⁶

{
  "actionType": "mail.search",
  "request": {
    "query": "",
    "limit": 4,
    "actionableOnly": true,
    "excludePromotional": true,
    "excludeAutomated": true,
    "excludeTransactional": true
  }
}

For one candidate, I used mail.thread_summary and mail.get_message to retrieve the available content:

{
  "count": 1,
  "threadCompleteness": {
    "messageCount": 1,
    "fullContentMessageCount": 1,
    "possiblyTruncated": false,
    "reconstructionStatus": "bounded_local_records",
    "provenanceStatus": "source_ids_available"
  }
}

The local thread contained one message from 22 September. It asked other people to record something and confirm a choice. That did not establish that I owed a reply. The later exchange needed checking first.³⁶

An assessment with a reason

I want four possible outcomes: probably with me, probably waiting for someone else, probably finished, or insufficient evidence. A flag such as needsReply can find a candidate. The content must support the assessment.

A deadline, appointment or commitment can then affect priority. That wider calendar-and-task round is not part of the small example above. I want to bring the existing reading functions together later in mail.followup_candidates.

Where did I leave off yesterday?

A day later, I sometimes remember the topic I was working on but not why I rejected an option. I want to recover the last decision, the open question and the document I can continue with.

I am still designing that route. It starts with a local time window and available traces: changed notes, dated files, browser history, conversations, tasks and repository activity.

Viewing a page is not making a decision. A currently open tab does not show what was open yesterday either. The route needs to distinguish viewed, changed, discussed and decided. The desired answer points back to the source of the last known state.

Connecting to my Mac

I want to keep using Apple Notes, my calendar and existing files. I therefore connect to apps already on my workspace. macOS can allow apps to control other apps with permission; the available functions differ by application.¹

For reading, I combine files, exports, app connections and, where useful, local SQLite databases or caches. A cache can lag behind, and an app update can change its format. An empty result does not automatically mean the information is absent.

A suitable API remains an option too. For changes, I use app functions or automation. Being able to read an internal app database is not a reason to write directly into it.

Notes on the device I have with me

Apple Notes already gives me a reading and editing app on my Mac, phone and iPad. Notes in iCloud can sync between those devices when they share an Apple Account and Notes is enabled. “On My Mac” notes are outside that route.¹⁵¹⁶¹⁷

That saves me building a note editor and mobile sync. Handoff also helps me continue in supported apps on another Apple device.¹⁸

My knowledge database does not sync automatically with it. An edit must reach the Mac and then pass through my own processing. The connection, sync and next processing round determine when it reaches the knowledge layer.

OpenClaw and my MCP connection

MCP, the Model Context Protocol, describes how an AI client uses exposed tools and context. My MCP server makes my own environment’s functions available. The protocol does not decide how I organise knowledge.²

OpenClaw is the gateway for clients, channels and agents. I build my source adapters, knowledge layer and working rules around it. Those additions are not automatically standard OpenClaw features.³

Which parts I build myself

Reusing the gateway

I use OpenClaw for the infrastructure that routes requests and organises execution. That lets me spend my time on the information I need: the right document, the right person and the history of an appointment.³

My own knowledge layer

I decide how sources, people, versions and decisions fit together. Source selection, search logic, corrections and my working rules sit behind my own MCP connection. I want to carry those choices to another AI client too.

Reading and executing

Reading a calendar and moving an appointment require different permissions. For actions with consequences, I want a concrete proposal and the approval my policy requires. OpenClaw provides execution approvals; app permissions and my adapters retain their own responsibilities.³⁷

Choosing another gateway later

I expect integration work when switching. What I want to retain is the sources, relationships, corrections and history in my knowledge base. That needs exportable structures, versions and recoverable backups. Changing the embedding model also requires its own migration.

Writing, typing and recording

My notes start in different places. I write by hand on reMarkable, with content and exports available through its desktop app. I organise the retrieval, extraction and connections to my other sources myself.⁴

PLAUD is an audio source. Its export options give me a route to my own processing, while the transfer can still depend on PLAUD software.⁵

I am also designing a notetaker for conversations on the Mac. I want these inputs to feed the same knowledge environment, so choosing a pen or a microphone does not determine where I have to search later.

The audio connection, import counts and my experience with sync are in the PLAUD article.

From a voice to a known speaker

My notetaker design starts with transcription. Whisper can run locally on Apple Silicon through whisper.cpp. The surrounding application has to organise audio segments and timestamps.⁶

NVIDIA Nemotron 3 Diarization is assigned the task of finding speaker changes. It supports streaming and offline processing and produces speaker labels. A label is not yet a name from my contacts.⁷

That needs a separate comparison with known voice profiles through speaker embeddings. An attendee list helps find candidates but does not establish whose voice it is. When uncertain, the identity stays unknown or needs confirmation.⁸

A confirmed speaker can then connect to a contact and the related email, appointment and previous notes. Transcription, diarization and identification remain separate components.

Reading a conversation again

After a brainstorm, I want ideas and trade-offs. After a technical meeting, I am more likely to look for decisions and responsibilities. I want the note format to suit the conversation and remain easy to correct.

Apple Notes is the intended reading app. It needs to show what was said in the conversation, what came from an earlier source and what the AI is proposing. My corrections should then return to the knowledge base with their source and version.

Looking back at a workday

Where was there space in my calendar?

On 4 October I retrieved the calendar for Tuesday 22 September 2026. This reconstructs that schedule; it is neither an automatically delivered morning briefing nor a time sheet. Client names have been replaced with project letters.³⁸

Within a chosen 08:30–17:00 work window, six timed blocks added up to six hours. One project block led directly into a demo. The open time was scattered through the day.

Open time is not always usable work time

As a planning proposal, I added fifteen minutes of demo preparation, fifteen minutes to wrap up and half an hour for lunch. Work and a break then took seven hours. Ninety minutes remained, split across three blocks:

That leaves room for a thirty- or forty-five-minute task. A task needing ninety uninterrupted minutes would require moving something. The allowances came from the proposal; they were not in the calendar.

The source behind the calculation

After a query covering 21–27 September, calendar.range retrieved Tuesday separately. The read used apple-calendar-sqlite, returned eight source rows and reported no truncation.³⁸

{
  "actionType": "calendar.range",
  "request": {
    "fromDate": "2026-09-22",
    "toDate": "2026-09-22",
    "limit": 20
  }
}

{
  "count": 8,
  "possiblyTruncated": false,
  "windowMode": "explicit_date_range",
  "sourceStatus": "searched",
  "networkUsed": false,
  "sourceDiagnostics": {
    "calendarAdapters": [
      "apple-calendar-sqlite"
    ]
  }
}

Six rows were work blocks. Two described the same multi-day informational event, so they were not counted as two extra appointments. Actual attendance would change the calculation.

The week query also showed 286 minutes of scheduled travel on Wednesday. That supported the suggestion to gather missing input on Tuesday. This retrospective did not include that morning’s email or tasks, and the calendar remained unchanged.³⁸

Connecting sources

My local knowledge base uses PostgreSQL with pgvector for vector search. Relevant passages can be retrieved before generating an answer: retrieval-augmented generation, or RAG.⁹

I also record relationships between people, conversations, documents, projects and decisions. Suppose a later email changes an earlier material choice. For “what applies now?”, I want both sources and the reason the choice changed. That is an example of the relationship I want to follow.

Each relationship needs a source, date and status: confirmed, inferred or uncertain. The graph helps with search, but a line between two records does not establish that the connection is true. I describe result selection further below.

Contact details from an email signature

A smaller example is a name or phone number in an email signature. I want a proposed address-book update that is first compared with what is already known.

Its origin needs to stay attached. A forwarded signature can belong to somebody else, and an old job title may have changed. Only after that check would I want to connect the contact with conversations, email and appointments.

Why I add my own connections

ChatGPT and Claude already combine apps and sources. I use those building blocks too. My choice to build an environment of my own is about where I keep relationships, source status, corrections and working rules.¹⁰¹¹

I want to decide which document version applies and when two sources refer to the same person. A standard connector can expose a source; my search layer decides how to select that information alongside other sources.

Trying a different model

I want to keep my sources, relationships and corrections outside a single chat session. Another client can then access them through a suitable adapter. Memory import and export, such as Claude offers, can also help with a switch.¹²

That does not produce identical answers. Models differ in their use of context and tools, so I need to try and check those behaviours again. Information left only in an old chat does not move automatically.

The same arrangement leaves room for local models on particular tasks. It does not make the whole environment offline: sync services, external models and channels such as Telegram still have their own connections.

From an answer to an action

From a retrieved source, I want to prepare a draft reply, open a note or suggest a task. The MCP setup includes specific functions for actions, memory updates and feedback.²⁸

Telegram is a channel for proposals and approvals in my setup. OpenClaw can route execution requests there for review. For a change, I want to see what I am about to allow; a simple permitted read need not interrupt me every time.¹³

Maintaining the connections

Apps and file formats change. When search fails, I want to know whether the cause is the source, sync or adapter. Technical feedback gets a separate maintenance route.

Through OpenClaw, I want to turn that report into a small repair task. Codex can investigate, edit and test relevant code through the CLI. Before a change reaches my working environment, I want to review what it changes and what it affects.¹⁴

After an edit, the original question needs checking again. A missing macOS permission, for example, needs a different fix from a changed database field.

Technical detail: from source to answer

This is how I separate the layers. The earlier examples show concrete reads. This overview also includes design choices for further development; not every search route runs for every question.

A. Interfaces and app access

An AI client discovers MCP tools through tools/list and uses tools/call. inputSchema defines input; an optional outputSchema defines structured output. My bridge translates calls to OpenClaw and source adapters. MCP transports such as stdio or Streamable HTTP are separate from OpenClaw’s WebSocket control plane.^1920283

Adapters can control scriptable apps through AppleScript or JXA, use native frameworks or read local data. NSWorkspace can identify the frontmost app; EventKit exposes calendars and reminders. The adapter must identify whether it returns a filename, metadata or content.²¹²²²³

With SQLite in WAL mode, recent data can still be in the write-ahead log. Copying only the main file is therefore not a general snapshot method. The reading route needs a consistent transaction or suitable backup within the service’s macOS permissions.²⁴²⁵²⁶

The examples include source labels such as spotlight_local, icloud_drive, microsoft-outlook-calendar-widget and apple_reminders. The availability of EventKit does not mean those particular reads used it.³³

B. Ingestion and derived data

Ingestion detects changes, extracts content and builds searchable representations. Text chunks, tables and relationships can be updated separately.

Sources → detect changes → extraction + source metadata
                         ├→ text chunks → embeddings
                         ├→ sections, tables, lists and content objects
                         └→ entities, relationships and observations
                            ↓
             search indices + versions + quality + freshness

PostgreSQL holds document records, content objects, entities, relationships and observations. Source IDs and versions connect those records to the original content. Chunks, embeddings and indexes are derived from it.⁹²⁷

When reprocessing, I want to update changed representations, mark old versions and avoid duplicate records. Deletions and withdrawn sources need to propagate too. A Notes correction must reach the Mac before ingestion can process it.¹⁵¹⁶

The generative model, embedding model and any neural reranker are separate components. A different embedding model can require reindexing; matching vector dimensions do not make two embedding spaces interchangeable.³⁰

C. Meaning, identity and time

The question determines intent, person, topic and time window. “Yesterday” becomes a local date; “last conversation” refers to conversation time. A stable contact ID is stronger evidence than a matching name or shared mailbox.

Source permissions, explicit person IDs and required date ranges constrain selection. A high semantic score must not override them.

D. Different search routes

Exact and full-text: names, terms, filenames and numbers. PostgreSQL full-text ranking is not automatically BM25.²⁹

Vector: content that discusses the same subject in different words. Similar meaning does not establish identity or an exact part number.⁹

Structured: sections, tables and lists with headings, row context and units. Graph: search through supported relationships, with limits on type and depth.

Memory and apps: preferences, corrections, email, calendars and conversations. The question determines the route. Current news needs an additional external source; a local index does not provide it automatically.

E. Combining and ranking results

I combine candidates by source identity and preserve their location and retrieval route. Copies of one source do not count as independent confirmation. Text scores and cosine similarities cannot simply be added together; rank fusion is one possible way to combine routes.⁹

Reranking reassesses retrieved candidates for the question. My scoring uses relevant terms, exact entities, provenance, relationships and time, among other signals. Duplicates, noise and conflicting versions can be ranked lower.²⁷

For “who did I last speak to?”, a dated conversation can belong above a recent newsletter. A separate cross-encoder is a possible additional step, not something established by the presence of a ranking score. The score itself is not a probability of truth.³⁰

F. Dates, versions and provenance

Event time, validity period, source modification and indexing time mean different things. An old decision can still apply; a new draft may not replace it yet. A refreshed index of a stale cache still contains stale information.

I group duplicates, versions and derived copies. An email, export and AI summary can share one origin. Near-identical versions still need comparison: the changed number may be the answer.

A claim needs a supporting passage, table cell, message or timestamp. A link to a document on the same subject is not enough.

G. The evidence package for the model

Selected material goes into a bounded context package. Passages retain the headings, versions and references needed to read them. Dates, units, negations and exceptions need to survive shortening.

Answerability asks whether the sources support the requested conclusion. Coverage asks which parts of the question are covered. Finding the appointment alone does not complete the preparation.

H. Searching again with a purpose

A missing piece can justify another query: resolving a transcript name, searching email for a project alias or retrieving later replies. The new results are then assessed again.

I bound that loop by rounds, time and tokens. If a round adds nothing, repeating the same broad query does little.

Question → intent + identity + time + constraints
         → routed hybrid retrieval
           [exact/FTS | vector | structured | graph | memory | apps/web]
         → candidate fusion → reranking
         → context package → answerability + coverage
         → answer, or targeted retrieval within budget

Throughout the pipeline:
source selection · freshness · validity · quality · dedupe · provenance

I. From an answer to a proposed action

An answer does not grant permission to send email, change an appointment or deploy code. Those actions follow their own permissions and approvals. App changes use suitable app functions.¹³³⁷

The combinations below are possible routes using the building blocks in my registry. They describe intended results; connected sources determine what an execution can deliver.²⁸

J. Finding the cause of a failure

I want visibility into source freshness, extraction coverage, index status, queues, graph delay and tool errors. That helps trace an empty result to its source or processing stage.

For changes in search quality, I want fixed, non-sensitive reference questions. Exact matches, valid decisions and the contribution of a second search round need different checks from counting documents.

After a repair, the original user question needs trying again. The client matters too: changing a server does not automatically refresh the tool definition known to the AI client.¹⁰

Why I spend time on this

I know which information I keep gathering again and which apps I want to use. Building the connections myself lets me start with my own questions. It also shows me where a tidy architecture still falls short in use.

A starting prompt for the project

The prompt below helps a coding agent first document the existing environment and the questions it should support. Decisions, instructions and one small working route follow. It is a starting point to adapt to your own situation.

Opening or copying it here runs nothing. Use it in the workspace you want examined, with the permissions appropriate to that task.

Starting with one question

If you want to try this, I would choose one question that keeps sending you into several apps. Preparing an appointment from your calendar, email and notes is one example. First check whether that one route finds the right sources.

Then add another source. Keep track of development time, maintenance, hardware and model use too. Your own knowledge layer is worthwhile only if you want to keep using and maintaining it.

What I want to work on next

The house and meeting questions show why I am building this. I can start with my own sources, their dates and their history. Next I want those connections to help when following up a conversation or resuming an earlier task.

I am happy to try a different model. I want to keep using my own context when I do.