Priyesh Digital Memory

When I recently reviewed my investment pipeline, it turns out that a big number of successful companies in my anti-portfolio are not there because I passed on them, but because I did not reconnect at the right time.

Averaging over 400 pitch meetings and panel discussions annually, I have come to realize that the human capacity for information handling and recall is a key bottleneck in early VC careers. Assuming the average meeting to investment ratio in our industry holds at roughly 50:1, that means 98% of the high-signal intellectual property I ingest is effectively archived. However, it wouldn't be a surprise if a specific insight from a founder meeting two years ago becomes the critical validation point for a portco pivot today. Yet, the standard toolset, scattered notions, static CRMs, and keyword queries, if any, fails to capture the context of those interactions.

There is a proliferation of "Chat with your Data" tools, but for a professional investor, these off-the-shelf solutions fail on two critical axes:

1.  Precision vs. Hallucination: In finance, an answer without a citation is a liability. I do not need an LLM to "summarize" a deal loosely; I need it to tell me exactly why I was concerned about the GTM strategy of a tool that was seeing great success in PLG motion from the call on May 14, 2023, with a link to the source paragraph. The system must prioritize contextual understanding over provenance over fluency.

2. The Privacy Bifurcation: A VC's reputation is built on trust. I cannot use an enterprise search tool that remotely risks leaking confidential deal metrics to a public interface. Pitch transcripts, ChatGPT queries are now as personal to me as my Google Photos.

This necessitated moving from evaluating the 100th software to building a system: a custom "Digital Memory" architected for zero-trust leakage and precise recall, the system closely tuned to VC workflows.

Which system approach would work best for a VC memory stack

The Temporal Blind Spot

Standard vector databases excel at semantic similarity but struggle with temporal continuity. If I ask, "How has my thesis on Agentic AI evolved since 2022?", a standard vector search might return the most relevant chunks regardless of time, mixing 2025 skepticism with 2024 optimism.

To solve this, the memory system implements a Temporal Graph. Each transcript segment is treated not just as a vector but is anchored to a timestamp. This allows the retrieval engine to re-rank evidence based on chronological flow, enabling the system to narrate the evolution of a thought process rather than just retrieving static facts.

The "Hallucination of Citation"

One of the non-negotiables is "trustworthy citations." I solve this by enforcing a strict Retrieval-Generation Separation. The generation layer is constrained to use only the context fragments provided by the retrieval layer. If the semantic search confidence score drops below a threshold, the system is programmed to explicitly state where the evidence is thin - rather than fabricating a bridge

Entity Resolution and Knowledge Graphs

Beyond vectors, the system builds a Knowledge graph. It extracts entities (Founders, Companies, Sectors) and builds relationships Entity -> RELATED_TO -> Entity). This is where tuning the memory bank specific to VC data sources and workflows yields the highest accuracy retrievals.

The Stack looks like

Ingestion: Aggregates transcripts (Otter), ideation (ChatGPT), and network data (LinkedIn).
Extraction (OpenAI + Instructor): I utilize gpt-4o-mini wrapped in the Instructor library. Instructor enforces Pydantic validation loops, driving success to 95%+.
Embedding (Google Gemini): text-embedding-004 for vectorization. It offers superior cost-efficiency and multilingual support (critical for India-US corridors)
Storage (Neo4j): Stores structured entity nodes and raw episodic memory nodes.
Temporal Graph (Graphiti by Zep): open-source Pythonframework for building temporally-aware knowledge graphs

Hybrid Retrieval: The "Zero-Trust" Approach

It would be a shame to have an unfit retrieval schema on top of a detailed knowledge graph. To ensure it misses nothing, I employ a hybrid retrieval strategy. If I query, "Climate founders I met at Demo Day," the system executes three parallel paths:

1.  Vector Similarity: Finds embeddings near "Climate" and "Demo Day."

2.  Keyword: Exact term matching for "Demo Day" (handling proper nouns).

3.  Graph Traversal: (Me)-[:MET]->(Person)-[:WORKS_AT]->(Company {vertical: climate}).The results are then synthesized using a generative reranker (specifically Gemini 2.5 Flash Lite).

As I built AI memory, I realized an incessant desire to build an MCP server on top of it and connect my memory to any AI-native application I evaluate and use. Gone are the days of cold starting a new AI application. I'm also ambitiously thinking of putting a few more blocks across CRM, meeting transcription, task management, and an external-looking AI chatbot: putting it all together to create an AI-native digital VC Priyesh. Will update as I start putting together different building blocks with the end goal of out automating myself