AI Summaries Are Changing Email and Communications Strategy
With the introduction of Apple Intelligence, millions of customers can now create AI-generated summaries of their emails. AI email summarization marks a fundamental shift in how organizations communicate. Communicators must now consider how artificial intelligence interprets their content.
At North Coast Communications, we wanted to understand what this change means for audiences, policymakers, and organizations that rely on email to share critical information. So, we ran a study. AI Email Summarization Using Apple Intelligence tests how small changes in content structure and hierarchy affect Apple’s AI summaries.
AI Email Summarization Using Apple Intelligence
Get Email Intelligence Updates
How the Study Worked
We tested how Apple Mail’s new AI summarization tool responded to changes in email structure and found that results varied across time and device. Using historical and policy-based texts, including the Gettysburg Address, the Magna Carta, Federalist No. 51, and a recent U.S. State Department press release, each document was summarized multiple times on different iPhones and at different times to establish a baseline for consistency. The baseline tests revealed that Apple Intelligence summaries are not consistent: identical emails summarized at different times or on different devices emphasized different parts of the same message. In routine communication this variability may go unnoticed, but in political, policy, or market contexts, even a low-probability error could dramatically reshape how a message is received.
We then examined how Apple Intelligence summarized each document under controlled variations in structure:
- The order of information (placing certain content higher up)
- The first sentence
- Email length
- The use of headings
What We Found
The results offer practical guidance for communicators adapting to AI-driven platforms:
- Hierarchy matters: Information placed at the top of an email is most likely to appear in the AI summary.
- First sentence effects are inconsistent: Changing or removing the first line altered the summary in fewer than half of all tests.
- Shortening reduces nuance: Condensed emails produced briefer summaries that omitted supporting context or detail.
- Headers can shape meaning: In some cases, Apple’s AI incorporated header text directly into the summary, reframing emphasis.
- Recognition introduces bias: The AI occasionally “recognized” famous texts (e.g., Gettysburg Address), generating summaries based on preexisting knowledge rather than the actual email content.
- Unexplained variability persists: Identical messages sometimes produced different summaries across time and device, underscoring the non-deterministic nature of AI summarization.
Why It Matters
For communicators, policymakers, and marketers, AI summaries represent a new layer of audience interpretation. If an AI misrepresents your message, it can affect readers’ perceptions of your organization, your campaign, or your brand.
This makes message fidelity – the alignment between what you write and what audiences receive – a communications priority.
As Apple, Google, and other platforms expand these tools, email design and writing will evolve alongside them. And so may regulation.
Looking Ahead
We believe understanding how AI interprets text is essential for anyone sending digital communications at scale. The next phase of our research will explore how to optimize email content in real time, helping communicators test and refine their messages before sending. Stay tuned as we continue to monitor how AI reshapes the digital communications landscape.
About North Coast Communications
North Coast Communications is a full-service public relations and public affairs consultancy based in Washington, DC, with offices in Manila. We design strategic communications campaigns using traditional and digital PR best practices for private-sector, nonprofit, and government clients. Learn more at www.northcoastcomms.com.