Why organisations must move beyond content production and build structured information systems that can be discovered, trusted and retrieved by both people and AI.
As content becomes abundant and search evolves into retrieval, competitive advantage is shifting away from publishing volume and towards the systems that organise, verify and distribute information. Organisations that invest in structured information, proprietary data and trusted knowledge systems will be better equipped to remain discoverable, authoritative and relevant in an AI-native web.
The Great Content Commoditisation
Search Is Changing Faster Than Most Organisations Realise
The Hidden Asset Most Organisations Have Ignored
What Modern Information Systems Actually Look Like
Why Trust Becomes the New Competitive Advantage
The Emergence of the Two-Track Web
A decade ago, competitive advantage on the web was often measured by publishing volume. More content meant more pages, more search visibility, and more opportunities to attract an audience. That equation is breaking down.
The economics of digital publishing are undergoing a structural shift. Generative AI has dramatically reduced the cost and effort required to create content, enabling organisations to produce articles, reports, product descriptions and marketing assets at unprecedented scale. Enterprise adoption is accelerating accordingly. Gartner reports that AI has moved from experimentation into operational deployment across a growing proportion of large organisations, while McKinsey's research suggests generative AI could contribute trillions of dollars in annual productivity gains by augmenting knowledge work across industries.
Yet the same technologies increasing content supply are simultaneously reducing the value of content itself.
As AI-generated material floods the web, information abundance is becoming the default state. Differentiation through publication volume is increasingly difficult when competitors can generate similar outputs at near-zero marginal cost. At the same time, the mechanisms that historically connected audiences with content are changing. Search engines are evolving from referral platforms into answer engines. Google's expansion of AI Overviews places machine-generated responses directly within search results, reducing the need for users to visit the originating source. Industry studies have linked these changes to significant declines in click-through rates and referral traffic for publishers and brands.
The implications extend beyond search performance. The Reuters Institute's Digital News Report 2025 documents a continuing shift away from traditional news websites and towards platform-mediated discovery, while also finding that audiences still place a premium on trusted and recognisable sources in an increasingly fragmented information environment. Publishers are increasingly concerned that AI summaries and conversational interfaces will further reduce direct traffic over the coming years.
This creates a paradox. Information has never been easier to produce, yet trustworthy information has never been more difficult to surface.
For senior publishing, media and marketing leaders, this is the strategic challenge of the next decade. The question is no longer how to create more content. AI has largely solved that problem. The more important question is how organisations preserve discoverability, authority and trust when machines increasingly mediate access to information.
This challenge is explored further in our The Future of Digital Publishing in the Age of AI analysis, which examines how AI is reshaping publishing economics, audience acquisition and information discovery.
In this environment, value migrates away from content production and towards the systems that organise, verify, structure and distribute information. As the web evolves from a collection of pages into a network of machine-readable knowledge, trusted information systems—not content volume—become the foundation of sustainable competitive advantage.
The challenge facing organisations is no longer producing content. It is preserving discoverability, authority and trust.
For more than two decades, digital strategy has largely been built on a simple assumption: publishing more content creates more opportunities for discovery, engagement and growth.
That assumption is becoming increasingly difficult to defend.
The fundamental economics of content production have changed. What was once constrained by time, expertise and budget can now be produced at near-zero marginal cost. As a result, content itself is losing scarcity. And when scarcity disappears, so does value.
The release of large language models has transformed content production from a labour-intensive activity into a largely automated process.
Systems developed by OpenAI, Anthropic and Google can now generate articles, reports, summaries, product descriptions and marketing assets in seconds. What previously required teams of writers, editors and subject specialists can increasingly be completed through AI-assisted publishing workflows.
The impact is already visible across the enterprise. McKinsey research suggests generative AI could add between $2.6 trillion and $4.4 trillion in annual economic value, with a significant proportion derived from productivity gains in knowledge work. Meanwhile, Gartner forecasts continued acceleration in enterprise AI adoption as organisations move from experimentation to operational deployment.
The consequence is not simply faster production. It is a structural increase in the global supply of content.
Every organisation now has access to tools capable of producing material at a scale that would have been economically impractical only a few years ago. The barriers that once differentiated high-volume publishers from everyone else have largely disappeared.
Content production is no longer a specialist capability. It is becoming a utility.
Economic value is often created through scarcity. When something becomes abundant, competitive advantage shifts elsewhere.
The web is now experiencing an unprecedented supply shock. Millions of AI-assisted articles, blog posts, landing pages and social media assets are published every day. Content generation has become dramatically easier, but audience attention remains finite.
This creates a classic imbalance between supply and demand.
As content volumes increase, differentiation becomes harder. Many organisations are drawing from the same publicly available information, using similar prompts and producing remarkably similar outputs. The result is a growing layer of informational redundancy across the web.
At the same time, information workers are facing increasing levels of cognitive overload. Research consistently shows that employees spend substantial portions of their working week searching for information, verifying sources and navigating fragmented knowledge environments. More content does not necessarily create more understanding. In many cases, it creates more noise.
This is the paradox of abundance: the easier information becomes to produce, the harder it becomes to find what matters.
The traditional response to declining visibility has often been to publish more.
Yet in an environment of content saturation, volume can become self-defeating.
As the supply of content expands, individual assets compete against a growing number of alternatives for the same audience attention. Engagement becomes fragmented. Organic reach becomes harder to sustain. Marginal returns decline.
The effects are also visible within search ecosystems. As AI-generated content proliferates, search engines face increasing challenges in identifying originality, expertise and trustworthiness. Search quality becomes more difficult to maintain when vast quantities of similar content compete for visibility.
This is one reason why Google is investing heavily in systems that evaluate expertise, authority and trust, while simultaneously shifting towards AI-generated answers and information retrieval models. The competitive battleground is moving beyond keyword matching and page-level optimisation.
The strategic implication is significant.
If every organisation can produce content at scale, content production can no longer be the primary source of competitive advantage. The organisations that succeed will not be those that publish the most. They will be those that provide the most trustworthy, verifiable and discoverable information.
Content is becoming a commodity.
Trust is becoming scarce.
And scarcity is where value accumulates.
Content is becoming a commodity. Trust is becoming scarce. And scarcity is where value accumulates.
The collapse in content scarcity is only one side of the challenge. The other is that the mechanisms used to discover information are being fundamentally redesigned.
For much of the commercial web's history, search engines acted as intermediaries. Their role was to connect users with relevant webpages. Visibility depended on creating content that could be indexed, ranked and clicked.
Today, that model is being replaced by something very different.
Search is evolving from a navigation system into a retrieval system. Increasingly, users are not being directed towards information. They are being given information directly.
For organisations that depend on visibility, authority and audience growth, the implications are profound.
The emergence of AI-generated answers represents the most significant shift in search behaviour since the introduction of mobile search.
Google's AI Overviews, Microsoft's AI-powered Bing experiences and a growing ecosystem of conversational answer engines are changing how users interact with information. Instead of reviewing multiple sources, comparing perspectives and visiting websites, users increasingly receive synthesised answers directly within the search experience.
This accelerates a trend that was already underway. Featured snippets, knowledge panels and rich results had already reduced the need for clicks. AI-generated responses extend that logic further by absorbing larger portions of the information journey.
Research from Seer Interactive found that organic click-through rates can decline significantly when AI Overviews are present, with some studies reporting reductions exceeding 60% for affected queries. What matters is not the precise percentage. What matters is the direction of travel.
The search result is becoming the destination.
At the same time, conversational discovery is reshaping user expectations. People are becoming accustomed to asking complex questions, refining them through dialogue and receiving contextual responses. Whether through Google, Bing, ChatGPT, Claude or other interfaces, discovery increasingly resembles retrieval rather than navigation.
The consequence is straightforward: being indexed is no longer enough. Information must be understandable, extractable and reusable by machines.
The search result is becoming the destination.
As search platforms answer more questions directly, referral traffic inevitably declines.
Publishers have been among the first industries to experience the effects. Analysis from WoodWing and other industry observers suggests that some publishers have experienced referral traffic reductions ranging from 62% to 97% for certain informational queries where AI-generated summaries satisfy user intent without requiring a site visit.
The challenge extends beyond news organisations.
Any business model built upon search-driven audience acquisition faces the same structural pressure. Fewer clicks mean fewer opportunities to build direct relationships, generate subscriptions, capture leads or monetise attention.
Research from Digital Content Next has also highlighted a broader fragmentation problem. Audiences increasingly discover information through social platforms, messaging apps, creator ecosystems, AI assistants and search interfaces simultaneously. Discovery is no longer concentrated within a small number of predictable channels.
At Digital Faction, we see similar patterns emerging across client portfolios. Organic impressions may remain stable or even increase, while click-through rates decline. Visibility is becoming detached from traffic. Organisations are appearing more often in search experiences but receiving fewer visits as platforms absorb a greater proportion of user interactions.
This represents a fundamental change in digital economics. Distribution is moving closer to the platform layer, while content creators lose control over the audience journey.
Ironically, despite advances in retrieval technology, many organisations remain poorly equipped for modern discovery.
The reason is not a lack of content. It is the quality of the underlying information architecture.
Most enterprise websites still rely on legacy search systems designed for document retrieval rather than knowledge discovery. Information is fragmented across departments, content management systems, databases and repositories. Metadata is inconsistent. Taxonomies are incomplete. Relationships between concepts are rarely defined in a structured way.
As a result, organisations often struggle to find their own information, let alone make it discoverable to external search engines or AI systems.
WAN-IFRA and other industry bodies have repeatedly highlighted the growing importance of structured content, metadata governance and semantic classification. These capabilities are becoming prerequisites for discoverability in an environment where machines increasingly interpret and retrieve information on behalf of users.
The critical point is that search has never really been about pages. It has always been about information.
Traditional search engines simply masked that reality by sending users to webpages. Modern retrieval systems expose it. They are designed to identify entities, relationships, facts and context rather than rank documents alone.
This is why many organisations are experiencing declining visibility despite producing more content. The bottleneck is no longer publication. It is retrieval.
Discovery is moving away from webpages and towards information retrieval. Organisations that continue optimising only for pages risk becoming invisible within the systems increasingly responsible for finding, interpreting and distributing knowledge.
Discovery is moving away from webpages and towards information retrieval.
The shift from search to retrieval changes what organisations must optimise.
When search engines primarily ranked webpages, publishing more content could compensate for weaknesses elsewhere. Retrieval systems are less forgiving. They depend on structured information, explicit relationships and reliable context.
This exposes a reality that has existed for years but was often hidden beneath page-level metrics: the most valuable digital assets are not webpages. They are the information systems that sit behind them.
The organisations best positioned for the next phase of the web are not necessarily those producing the most content. They are the ones that have invested in the systems that organise, connect and govern knowledge.
One of the most persistent misconceptions in digital strategy is treating content and knowledge as interchangeable concepts.
They are not.
A document is a container. A report, article, PDF or webpage is simply a format for presenting information.
Content is the expression of information within that container.
Information consists of facts, observations, data points and descriptions that can be communicated or stored.
Knowledge emerges when information is organised, connected and contextualised in ways that enable understanding and decision-making.
Most organisations are rich in documents and content but poor in accessible knowledge.
Consider a publisher with thousands of articles covering a specialist sector. The expertise exists. The information exists. Yet if the organisation cannot consistently identify entities, connect related concepts, surface authoritative sources or expose relationships between topics, that knowledge remains trapped within individual documents.
Historically, this limitation was manageable because humans performed the work of interpretation. Readers navigated archives, followed links and assembled context themselves.
Modern retrieval systems work differently. They seek structured information that can be retrieved, connected and reused automatically.
This is why publishing more content no longer guarantees greater visibility. Knowledge that cannot be understood by machines becomes increasingly difficult to discover.
More than twenty years ago, Sir Tim Berners-Lee proposed a vision for the future of the web that extended beyond linked documents.
His concept of the Semantic Web imagined a network in which information itself could be understood by machines. Rather than simply connecting pages through hyperlinks, computers would understand the meaning of the information contained within them.
At the time, the idea was often viewed as technically ambitious and commercially distant.
Yet many of the foundational components survived.
Standards developed through the W3C introduced frameworks for describing entities, relationships and meaning. Linked data principles established methods for connecting information across systems. Later, initiatives such as Schema.org created shared vocabularies that allowed organisations to describe people, places, products, organisations and events in machine-readable formats.
Meanwhile, public knowledge infrastructures expanded rapidly. Wikidata has grown into one of the world's largest openly accessible knowledge graphs, containing hundreds of millions of structured statements describing entities and their relationships. Its importance lies not simply in scale, but in demonstrating how machine-readable information can be organised, maintained and reused across countless applications.
What appeared to be a niche vision is increasingly becoming the architecture of the modern web.
The Semantic Web did not fail.
It arrived through AI.
The Semantic Web did not fail. It arrived through AI.
Perhaps the clearest example of this shift is Google's Knowledge Graph.
Before its introduction in 2012, search largely relied on matching keywords to documents. Results were determined primarily by signals associated with webpages and links.
The Knowledge Graph introduced a different model.
Instead of focusing exclusively on words, Google began identifying entities and the relationships between them. A search for "The Economist", for example, no longer required matching keywords across pages. Google could recognise an organisation, understand related people, products, publications and locations, and surface information accordingly.
Structured knowledge became part of the search engine itself.
As structured data specialist Aaron Bradley has argued, modern search is increasingly an "index of things" rather than simply an index of pages.
The implications extend far beyond Google. The same entity-centric principles now underpin recommendation systems, digital assistants, retrieval systems and large language models.
Modern retrieval systems are fundamentally different from traditional search engines.
They do not merely retrieve documents. They attempt to retrieve meaning.
To achieve this, they rely on three core elements:
A page about renewable energy, for example, contains words. A knowledge system identifies companies, technologies, regulations, researchers, markets and the relationships connecting them.
That distinction is critical.
Machines can process content. They can retrieve information. But they generate reliable understanding only when information has been structured into meaningful relationships.
This is why knowledge graphs, taxonomies, metadata frameworks and structured content models are becoming strategic assets. They provide the context modern retrieval systems require.
The organisations that thrive in the next era of the web will not be those with the largest content archives.
They will be those that transform information into knowledge that machines can understand.
Because AI systems do not retrieve pages.
They retrieve information.
AI systems do not retrieve pages. They retrieve information.
If content is no longer the primary source of competitive advantage, what replaces it?
The answer is not a single technology. It is an architecture.
Over the past two decades, many organisations have invested heavily in content management systems, digital asset platforms and publishing workflows. These investments improved content production, but they rarely addressed the deeper challenge of organising knowledge in ways that machines can understand.
The organisations succeeding in an AI-native environment are building layered information systems rather than isolated content repositories. These systems transform information from a collection of documents into a structured, governable and retrievable asset.
At Digital Faction, we describe this progression through an Information Maturity Model:
Each level builds upon the one before it.
Every modern information system begins with structured content.
For many organisations, content remains trapped within webpages, PDFs, reports and documents that are optimised for human consumption but difficult for machines to interpret. Structure changes that.
Structured content separates information from presentation. Rather than treating an article as a block of text, information is organised into defined components such as authors, topics, entities, publication dates, locations, products and categories.
This is where editorial metadata becomes strategically important.
Metadata provides context about information. It identifies what something is, who created it, when it was published and how it relates to other assets. Without metadata, information becomes difficult to classify, retrieve and reuse.
Schema markup extends this concept externally. Using standards such as Schema.org, organisations can communicate machine-readable meaning to search engines, AI systems and knowledge graphs. Instead of merely publishing a page, they describe the entities, attributes and relationships contained within it.
Content modelling completes the foundation by defining how information should be structured across the organisation. This ensures consistency between teams, channels and platforms.
The result is a shift from publishing content to publishing structured information.
Structure alone is insufficient.
Information must also be organised.
Taxonomies provide the classification systems that allow information to be grouped consistently. They establish controlled vocabularies that reduce ambiguity and improve discoverability.
For example, should an organisation classify content under "AI", "Artificial Intelligence" or "Machine Learning"? Without governance, different teams often use different terminology, creating fragmentation and retrieval problems.
Taxonomies create consistency.
Ontologies extend this concept further by defining relationships between concepts. They describe how entities connect and interact within a domain.
A taxonomy may define that a company belongs to the category "Technology Vendor". An ontology defines that the company develops products, serves customers, operates within industries and competes with other organisations.
This distinction is crucial because modern retrieval systems increasingly rely on relationships rather than keywords.
Information becomes more valuable when its connections are explicit.
If taxonomies provide classification, knowledge graphs provide context.
A knowledge graph connects entities and relationships into a structured representation of organisational knowledge.
Rather than storing information as isolated documents, a knowledge graph creates a network of connected concepts.
Customers connect to products.
Products connect to industries.
Industries connect to regulations.
Regulations connect to research.
Research connects to subject-matter experts.
This interconnected model enables something traditional content systems struggle to provide: contextual retrieval.
Knowledge graphs also support entity resolution—the ability to recognise that different references describe the same thing. A person, organisation or product may appear in multiple systems under different names or formats. A knowledge graph consolidates those references into a unified entity.
This capability has become increasingly important as organisations seek to make proprietary knowledge available to retrieval systems and AI applications.
The strategic value is straightforward. Content repositories store information. Knowledge graphs organise understanding.
Content repositories store information. Knowledge graphs organise understanding.
Once information has been structured and connected, it can be retrieved intelligently.
This is where Retrieval-Augmented Generation (RAG), vector search and GraphRAG enter the architecture.
RAG systems improve AI outputs by retrieving relevant information from trusted knowledge sources before generating responses. Instead of relying solely on model training data, they ground outputs in organisational knowledge.
Vector search enables semantic retrieval by identifying information based on meaning rather than exact keyword matches. This allows systems to retrieve relevant content even when users phrase questions differently from the original source material.
Hybrid retrieval combines semantic and keyword-based approaches, improving both precision and recall.
The next evolution is GraphRAG.
Research from Microsoft has demonstrated that graph-enhanced retrieval can improve performance for complex, multi-hop questions by incorporating relationships between entities rather than relying solely on document similarity. Instead of retrieving isolated passages, GraphRAG retrieves context.
At the same time, platforms such as MongoDB are integrating vector search directly into enterprise data architectures, reflecting the growing demand for retrieval-native information systems.
The direction is clear: retrieval is becoming an infrastructure layer rather than an application feature.
Reuters and the Associated Press offer an instructive example of this evolution.
Both organisations possess vast archives of structured content, editorial metadata and proprietary datasets accumulated over decades. Increasingly, these assets are being positioned not simply as publishing outputs but as machine-readable information resources that can support retrieval systems, licensing agreements and AI applications.
The strategic asset is not the article alone.
It is the structured information system behind it.
The final layer is the most important.
Without governance, every preceding layer deteriorates over time.
Information systems require stewardship. Data quality must be monitored. Metadata standards must be enforced. Taxonomies must evolve. Entity models must be maintained.
Equally important is provenance and lineage.
As AI-generated content proliferates, organisations need mechanisms to demonstrate where information originated, how it has changed over time, who validated it and how it entered a retrieval workflow. Emerging standards from OASIS Open, together with regulatory frameworks such as the EU AI Act, increasingly emphasise transparency, traceability and accountable data governance.
The NIST AI Risk Management Framework reinforces the same principle. Trustworthy AI systems depend on trustworthy information systems. Reliability is not created at the model layer; it is created through governance, stewardship and data quality management.
This is where the FAIR principles become strategically relevant. Information that is Findable, Accessible, Interoperable and Reusable is inherently better suited to retrieval, citation and machine consumption.
The organisations creating durable advantage are not merely publishing content. They are building governed knowledge assets that retain integrity as they move across channels, platforms and agent ecosystems.
This is ultimately what separates a content repository from a strategic information asset.
The future information stack is:
Content → Metadata → Entities → Knowledge Graph → Retrieval Layer
Content → Metadata → Entities → Knowledge Graph → Retrieval Layer
The defining challenge of the AI era is not generating information. It is determining which information can be trusted.
As content becomes abundant and discovery increasingly relies on automated retrieval systems, value shifts towards organisations capable of providing reliable, verifiable and authoritative information. The competitive advantage is no longer publishing more content than competitors. It is becoming a source that machines can trust.
Despite rapid advances in large language models, the underlying challenge remains unchanged: output quality depends on input quality.
Hallucinations, factual inaccuracies and fabricated citations are not simply model failures. They are often symptoms of weak information foundations. When retrieval systems draw upon incomplete, outdated or poorly governed information, unreliable outputs become inevitable.
Equally problematic are provenance failures. In many AI-generated responses, users cannot easily determine where information originated, whether it has been verified or who is accountable for its accuracy. This lack of transparency creates risk for publishers, brands and audiences alike.
The spread of misinformation further compounds the problem. As content generation costs approach zero, the volume of unverified information continues to expand. The difficulty is no longer access to information. It is confidence in its reliability.
This is precisely why data quality, provenance and governance have become strategic concerns rather than technical considerations.
The National Institute of Standards and Technology (NIST), through its AI Risk Management Framework, identifies trustworthiness as a foundational characteristic of effective AI systems. Reliability, transparency, validity and accountability are not optional features. They are prerequisites for sustainable adoption.
Organisations that cannot demonstrate these qualities will increasingly struggle to secure visibility within retrieval-driven ecosystems.
This shift helps explain why established information providers are becoming more valuable, not less.
Organisations such as Reuters, the Associated Press (AP) and the BBC have spent decades building editorial processes designed to verify information before publication. Those systems of verification are now becoming machine-readable assets.
AI platforms, retrieval systems and enterprise knowledge applications require trusted source material. They need information that is accurate, attributable and consistently maintained.
In this environment, institutional credibility becomes a form of infrastructure.
The Reuters Institute's research consistently demonstrates that trust remains one of the strongest predictors of audience engagement with news and information. While platforms, formats and distribution channels continue to change, confidence in the source remains remarkably durable.
This principle increasingly extends beyond human audiences.
Retrieval systems favour sources with strong authority signals, clear provenance and demonstrable expertise because those characteristics improve confidence in the resulting output.
The implication for organisations is significant: authority is no longer simply a brand attribute. It is a retrieval advantage.
The most valuable information assets are often those competitors cannot replicate.
First-party data, proprietary research programmes and audience intelligence datasets create unique informational advantages that generic content cannot match. They provide original evidence rather than commentary on evidence generated elsewhere.
This distinction matters because retrieval systems increasingly reward information gain.
A thousand articles repeating the same insight add little value. A proprietary dataset, original survey or exclusive research programme creates knowledge that exists nowhere else.
Many organisations still view research as a marketing activity.
Increasingly, it should be viewed as infrastructure.
The most effective projects Digital Faction encounters are rarely built upon larger content libraries. They are built upon unique information assets: audience studies, sector benchmarks, operational datasets and expert knowledge repositories that competitors cannot easily reproduce.
As retrieval systems mature, these assets become increasingly important because they provide both differentiation and trust.
The strategic shift is profound.
For decades, search rewarded visibility.
The next generation of discovery systems will reward credibility.
In a retrieval-first web, trust becomes machine-readable. Provenance becomes measurable. Authority becomes computational.
Trust is no longer merely a reputational benefit.
It is becoming a ranking signal for machines.
In a retrieval-first web, trust becomes machine-readable. Provenance becomes measurable. Authority becomes computational.
If trust is becoming machine-readable, information must become machine-readable too.
This is the logical next stage in the evolution of the web.
For most of the internet's history, organisations designed information primarily for human consumption. Search engines helped users find relevant pages, but people remained responsible for interpreting, evaluating and connecting information themselves.
Today, an increasing share of that work is being performed by retrieval systems, AI assistants and autonomous agents.
The consequence is not the disappearance of the human web. It is the emergence of a second layer alongside it.
Information must now serve two audiences simultaneously: people who read it and machines that retrieve, evaluate and distribute it.
The human web remains essential.
People will continue to read articles, watch videos, listen to podcasts and engage directly with brands, publishers and institutions. Design, storytelling, persuasion and user experience will continue to matter because trust is ultimately established through human relationships.
Alongside this familiar layer, however, a second ecosystem is taking shape.
AI assistants, enterprise copilots, retrieval systems and autonomous agents increasingly gather, evaluate and synthesise information on behalf of users. Instead of visiting multiple websites, users ask a question and receive a consolidated answer generated from numerous sources.
In this environment, visibility depends not only on whether a human can find your content, but whether a machine can understand it.
The implications are profound. Organisations are no longer competing solely for attention. They are competing for inclusion within machine-generated answers, recommendations and decisions.
This broader shift is examined in How Artificial Intelligence Is Changing Online Platforms , which explores how AI is transforming discovery, recommendation systems, commerce and platform economics.
Discovery becomes less about webpages and more about information accessibility.
Discovery becomes less about webpages and more about information accessibility.
This transition is driving demand for machine-readable information architectures.
The organisations most likely to succeed are those that treat content as structured data rather than isolated documents.
Emerging standards are accelerating this shift. Model Context Protocol (MCP), championed by organisations including OpenAI and Anthropic, provides a framework for connecting AI systems directly to trusted information sources. Rather than relying exclusively on static web crawling, agents can retrieve information through governed, structured interfaces.
Similarly, llms.txt has emerged as an early attempt to help publishers communicate with AI systems, providing guidance on how content should be discovered and interpreted.
Structured APIs, knowledge graphs and machine-readable metadata are becoming equally important. These mechanisms allow information to move directly into retrieval systems while preserving context, provenance and attribution.
The strategic question therefore changes.
Instead of asking, "Can people find this content?", organisations increasingly need to ask, "Can machines understand and use this information?"
As discovery evolves, optimisation evolves with it.
Traditional search engine optimisation focused on rankings, keywords and click-through rates. While these disciplines remain relevant, they are no longer sufficient in a retrieval-first environment.
A new discipline is emerging: Generative Engine Optimisation (GEO).
The objective of GEO is not simply visibility within search results. It is discoverability within answer generation systems.
That requires different signals.
Citation optimisation becomes critical because retrieval systems favour sources that can be confidently attributed. Confidence signals such as author expertise, provenance metadata, structured entities, organisational authority and evidence-backed claims become increasingly important.
In effect, organisations must optimise not only for indexing but for retrieval confidence.
Organisations must optimise not only for indexing but for retrieval confidence.
One of the clearest examples comes from The Economist.
Under the leadership of executives including Josh Muncke, the organisation has publicly explored how information products must evolve for a world in which agents increasingly mediate discovery. The emerging vision is not to abandon traditional publishing but to supplement it with machine-readable information products designed for retrieval, citation and reuse.
This reflects a broader industry shift already recognised in foresight work from the UK's Digital Regulation Cooperation Forum (DRCF): future digital ecosystems will be shaped by interactions between humans, AI systems and autonomous agents.
The organisations that thrive in this environment will not choose between audiences and machines.
They will build information systems capable of serving both.
The future internet is not replacing the human web.
It is adding an agent web alongside it.
The winners will be those who understand that both now matter.
The strategic challenge facing organisations is no longer how to produce more content.
It is how to build information assets that become more valuable over time.
For two decades, digital strategy largely rewarded scale. More pages, more articles, more campaigns and more content often translated into greater visibility. That equation is breaking down. As content creation becomes increasingly abundant, the value of any individual asset declines.
The organisations creating long-term advantage are taking a different approach. They are investing in information systems that accumulate context, strengthen trust and improve retrieval performance with every new contribution.
In other words, they are building systems that compound.
While technologies will continue to evolve, the strategic foundations are remarkably consistent.
Together, these priorities form a practical roadmap from content production towards knowledge infrastructure.
Many traditional competitive advantages are becoming easier to replicate.
Content volume is no longer scarce.
Publishing technology is no longer scarce.
Even content production expertise is becoming widely available.
What remains difficult to copy are the assets that sit behind the content.
Deep expertise creates interpretation and judgement that generic systems cannot easily reproduce.
Proprietary datasets create unique knowledge that competitors cannot access.
Audience trust creates confidence, attention and engagement that cannot be purchased on demand.
Knowledge systems create organisational memory that improves retrieval, discovery and decision-making over time.
Most importantly, these assets reinforce one another.
Expertise generates better research.
Research strengthens trust.
Trust attracts audiences.
Audience engagement produces new insights.
Those insights enrich the underlying information system.
This creates a compounding cycle that becomes increasingly difficult for competitors to disrupt.
Expertise generates better research. Research strengthens trust. Trust attracts audiences. Audience engagement enriches the information system.
That is why the future competitive moat is not content itself. It is the information architecture that organises, validates and distributes knowledge at scale.
The organisations that win in the next decade will not be those that publish the most content.
They will be the organisations that own the most trusted information systems.
The most significant shift underway is not technological. It is architectural.
The web is moving from pages to entities, from search to retrieval, and from publishing to knowledge systems.
For much of the internet era, visibility depended on creating content and making it discoverable through search. Success was measured by rankings, traffic and publishing output. That model is increasingly being disrupted by retrieval-based systems that prioritise context, relationships, provenance and trust over the existence of a webpage alone.
Machines do not navigate information in the same way people do. They retrieve facts, identify entities, evaluate sources and assemble answers from multiple datasets. In this environment, information architecture becomes as important as content creation itself.
This shift carries a profound business implication.
Content abundance destroys content scarcity.
When content can be produced at near-zero marginal cost, volume ceases to be a competitive advantage. Publishing more becomes less valuable when everyone can do the same.
At the same time, trust becomes increasingly scarce.
Reliable information, transparent provenance, subject-matter expertise and authoritative datasets become more valuable precisely because they are harder to replicate. As retrieval systems become central to discovery, trust evolves from a brand attribute into a machine-readable signal.
That changes where organisations should invest.
The next generation of competitive advantage will not be created through publishing velocity. It will be created through information quality.
Organisations that lead will invest in structured information, knowledge graphs, metadata governance, proprietary research and direct audience relationships because these assets improve every downstream activity. They strengthen retrieval accuracy, increase citation likelihood, enhance discoverability and reinforce trust.
The Information Maturity Model outlined in this article provides a useful lens for assessing readiness. Organisations operating at the level of content repositories and structured publishing may continue to compete effectively today, but the long-term advantage lies with those building retrieval-ready and agent-native information systems. In a world where machines increasingly mediate discovery, information architecture becomes a strategic capability rather than a technical consideration.
This is the deeper significance of the shift from pages to entities.
Pages are outputs.
Information systems are infrastructure.
Pages are outputs. Information systems are infrastructure.
The next decade will not be won by organisations that publish the most information. It will be won by organisations that build the most trusted systems for organising it.
In an AI-native web, content is no longer the asset. The information system behind it is.
Digital Faction Limited is a UK digital products company focused on digital publishing, artificial intelligence, search technologies, analytics, automation, research and online platforms.
Learn more about our work, topics and research initiatives.