Transcription, speaker labels and identity

The animation explains the architecture; no live actions are executed.

I am still investigating the NVIDIA runtime on my Mac.

Conversation audio

The recording is the source for transcription and diarization.

Whisper · local

• What is being said?
• Transcript + timestamps

Whisper produces text with timestamps. whisper.cpp provides local execution on Apple Silicon.

Nemotron 3 Diarization

• Sortformer: who speaks when?
• Speaker labels, not yet names

Nemotron 3 Diarization produces timed segments with speaker labels, without establishing contact names.

Transcript + speaker labels + audio references

Timestamps connect text, speaker segments and the original audio.

Voice profile

• Audio embeddings with known profiles

Speaker embeddings help compare known voice profiles. A possible match needs assessment.

Contact match

• Confirm identity or leave unknown

With sufficient evidence, a contact can be linked. Otherwise the identity remains unknown.

Email + calendar

• Link the exact contact to correspondence

A confirmed contact helps retrieve the related email and appointments.

Graph + context

• Projects, notes and earlier decisions

I want to retrieve relevant earlier notes, projects and decisions as background.

Context-aware template → Apple Notes

• Said in the conversation
• Known from earlier sources
• Proposed by the AI

The note needs to distinguish the conversation, earlier sources and suggestions.

Connections

• Conversation audio → Whisper · local
• Conversation audio → Nemotron 3 Diarization
• Whisper · local → Transcript + speaker labels + audio references
• Nemotron 3 Diarization → Transcript + speaker labels + audio references
• Transcript + speaker labels + audio references → Voice profile
• Voice profile → Contact match
• Contact match → Email + calendar
• Email + calendar → Graph + context
• Graph + context → Context-aware template → Apple Notes

How to read this

1. The recording is the source for transcription and diarization.
2. Whisper produces text with timestamps. whisper.cpp provides local execution on Apple Silicon. Nemotron 3 Diarization produces timed segments with speaker labels, without establishing contact names.
3. Timestamps connect text, speaker segments and the original audio.
4. Speaker embeddings help compare known voice profiles. A possible match needs assessment.
5. With sufficient evidence, a contact can be linked. Otherwise the identity remains unknown.
6. A confirmed contact helps retrieve the related email and appointments.
7. I want to retrieve relevant earlier notes, projects and decisions as background.
8. The note needs to distinguish the conversation, earlier sources and suggestions.