For AI labsPartnersTrustResearch
← All posts

AI labs5 min read

Why agents need workplace data, not more web text

The public web records finished work, not the process behind it. Agents that act inside companies need examples of state, handoffs and consequences that only workplace data contains.

Language models learned to write by reading the public web. Agents that work inside companies have to learn something the web barely records: how work moves between people, tools and decisions over weeks and months. Adding more web text will not close that gap. Workplace data can.

The web shows finished work, not the work itself

Most of what is public is the end product of a process. A blog post announcing a feature. A polished help-center article. A release note, a tidy README, a press announcement about a new customer. The steps that produced them are almost never published: the debate in a channel, the ticket that was reopened twice, the email asking finance whether the budget covered it, the pull request that was rejected and rewritten.

That is fine for a model whose job is to answer questions or draft text. It is a real limitation for an agent whose job is to take part in the process. An agent asked to triage a support queue, prepare a month-end close or ship a small fix needs to know what the intermediate states look like, not only the polished output at the end.

Put differently, the web is a library of answers. Agents need a record of how answers get made, including the wrong turns.

What an agent is actually asked to do

Consider the tasks teams want agents to take on. Pick up a ticket, find the thread where the bug was first reported, check what was already tried, write the change, open a pull request, respond to review comments and tell the customer it is fixed. Or reconcile an invoice against a contract, notice that the discount was applied twice, flag it to the right person and follow up when they go quiet for a week.

None of these tasks lives in one document. Each one crosses several tools, stretches over days, and depends on other people who have their own priorities. The skill being tested is less about writing and more about judgment: what to read first, what to ignore, who to ask, and when the job is actually done.

A model trained mostly on web text has seen plenty of descriptions of this kind of work. It has seen very few examples of the work itself, with its real sequence and its real loose ends.

Four things workplace data contains that the web does not

Operational records from a real company carry signals that are close to absent from public text:

  • State. A ticket has a status, an assignee, a priority and a history of every change. An agent has to read state and update it correctly. Public text rarely shows state at all, and almost never shows it changing over time.
  • Handoffs. Work passes from sales to onboarding, from support to engineering, from engineering back to the customer. The handoff is where context gets lost, and it is where agents tend to struggle most.
  • Consequences. In a real archive, a decision is followed by what happened next: the deploy that broke, the customer who renewed, the invoice that was paid late. That gives you something to grade against other than how plausible the agent's text sounds.
  • Norms. Every team has unwritten rules about who approves what, how urgent "urgent" is, and when to escalate. They show up only in the record of how people actually behaved.

These are the ingredients of the trajectories and environments labs are now building. For a broader map of which records hold them, see what counts as operational data.

Why more of the public internet does not substitute

It is tempting to think scale will fix this: crawl more forums, more public repositories, more question-and-answer sites. Public repositories do contain real engineering work and are genuinely useful. But open-source projects run differently from a company with paying customers, deadlines, budgets and a sales team. Forum posts describe workplace problems from the outside, after the fact, from one person's point of view.

Synthetic workplaces can fill specific gaps, but generated data tends to reproduce the patterns the generator already knows. Real companies are messier and more varied than anything a prompt will invent.

There is also a measurement problem. Public text is likely to be in training sets already, which makes it a weak foundation for evaluations. Private workplace data has not been crawled, so tasks built from it are far less likely to have leaked.

What makes workplace data usable

Workplace data is only useful if it holds together and if it protects the people in it. Done well, those two requirements support each other.

The key is consistent tokenization. Every person, company, address, email, phone number and account becomes a token that stays the same across every tool. PERSON_014 in Slack is PERSON_014 in email and in Jira, so an agent can follow one person's work across systems while no one can learn who that person is. Credentials, API keys, card numbers and government IDs are removed outright. HR, payroll, medical and legally privileged folders are excluded by default, and direct messages stay out unless the company chooses to include them and has the right to.

Connection depth matters more than raw volume. A modest company whose projects can be traced from chat to ticket to code to customer email is usually more useful than a much larger pile of disconnected messages. The licence terms matter too: training and evaluation only, with no marketing, profiling, contact or re-identification of anyone in the data.

Where this data comes from

Workplace data sits inside companies, both operating ones and those winding down. It does not come to market on its own. Someone has to find the right companies, reach the person who can sign, and run a process that respects the business and its employees.

That is the work we do. A lab sends a sourcing brief covering sector, headcount, tool stack, years of history, regions, and whether it wants operating or winding-down companies. We build a target list from company and technographic data, and our onshore sales reps call the people who can sign. The data is then evaluated, anonymized in-house and delivered on a rolling basis.

If your team is building agents and needs a specific mix of real workplace history, send us a sourcing brief.

Need company data for your models?

Send a sourcing brief and Martal Group's outbound team will find the companies that fit it.