Field Note · Applied AI and system ownership
A Family Office Needs Trusted Context Before More AI Tools
A family office can have excellent documents, excellent advisers, and excellent AI tools and still produce a bad answer if the system cannot tell which facts are current, who may see them, which professional question is being asked, and who is allowed to act.
A thinking frame by Andrew Moss
The questions I get
Usually some version of these:
- Where should AI go first in a family office?
- Can one AI assistant work across legal, tax, finance, investment, and relationship information?
- What must be in place before the tools become useful?
What a lot of people seem to think
The family should choose the most capable model, connect every document, and let one assistant make the information easier to use.
How I look at it
I would start with trusted context and permissions, not the model. Family information is unusually interconnected and unevenly shareable. The system needs source provenance, current facts, role-based access, adviser boundaries, decision history, and explicit human authority before broader intelligence becomes responsible leverage.
Why the decision matters
The cost is rarely confined to the line item.
If the sequence is wrong
The system can combine stale, inconsistent, privileged, private, or out-of-scope information into a fluent answer that nobody is actually authorized to rely on.
If the sequence is right
The family can retrieve the right context for the right person, prepare better specialist questions, preserve privacy, and keep consequential decisions with accountable humans.
How reversible is it?
Low after sensitive information is exposed, an adviser boundary is crossed, or an action is taken from the wrong version of the facts.
The short answer
Build the context and permission layer first.
Inventory authoritative sources. Identify people, entities, assets, documents, advisers, decisions, and relationships. Mark access, sensitivity, currency, and professional ownership. Start with bounded preparation and retrieval use cases. Add recommendations or actions only after provenance, evaluation, approval, and audit are real.
A useful pictureA powerful telescope does not fix the wrong map.
Better models can see more. They still need to know which territory is current, which labels are trusted, which areas are restricted, and who decides where the family is going.
Move fromMore connected tools→Move towardTrusted, permissioned family context
The order I would use
Take the right steps in the right order.
- 01
Inventory authoritative sources
Identify the current documents, systems, people, and advisers that own each kind of fact.
- 02
Map permissions and sensitivity
Define who may see family, trust, tax, legal, investment, lending, operating, and relationship information.
- 03
Record provenance and currency
Keep the source, date, version, and unresolved conflict visible rather than flattening everything into one memory.
- 04
Start with bounded preparation
Use AI to retrieve, compare, organize, and prepare questions before asking it to recommend or act.
- 05
Add human-approved loops
Require the relevant person or professional to verify the context and approve any consequential next step.
- 06
Measure the outcome
Track whether the system improved preparation, reduced rework, preserved privacy, and helped the family reach better coordinated decisions.
Questions worth answering
Before the next irreversible move:
- Which source owns this fact today?
- Who is allowed to see it, and for what purpose?
- Which qualified professional must interpret it?
- What action remains exclusively human?
- How will a correction replace a stale assumption without erasing history?
What not to do
Do not connect everything before deciding who should know what.
Do not let one convenient chat flatten professional, fiduciary, family, and privacy boundaries. Do not treat model confidence as source authority. Do not allow an AI system to transmit, invest, sign, disclose, or instruct a professional without explicit human control.
Keep the perspective
The valuable asset is not the chat window.
It is the trusted, permissioned context that helps the family and its advisers understand the same decision without exposing more than each person needs.
The boundary
What still depends on the facts
Family-office status, fiduciary duties, privacy, professional privilege, investment advice, tax, trust, lending, security, and data obligations depend on the actual structure, systems, jurisdictions, and use case.
Independent sources
Useful primary material
These sources support the public frame. They do not replace the private facts or the accountable professional.
Common follow-up questions
Should a family office connect every document to one AI system?
Not by default. Access, sensitivity, purpose, provenance, and professional boundaries should be designed before broad connection.
What is a safer first use case?
Bounded retrieval, comparison, meeting preparation, or question preparation from approved sources, with visible provenance and human review.