# THE AI / SEARCH VISIBILITY SWEEP - BUILDER INSTRUCTION Give this file to an AI coding agent (or a developer) together with: the site URL or repo, and the stack if known. The agent audits first, reports, then implements what is missing. Stack-agnostic. --- ## How to work through this file 1. **Phase A - Audit.** Run every check. Record PASS / FAIL / N/A per item in a findings table. Do not fix anything yet. 2. **Phase B - Report.** Output the findings table plus your implementation plan. Items marked [CONTENT] need owner input - propose wording, wait for approval, never invent business facts. 3. **Phase C - Implement.** Fix approved items. Smallest change that passes the check. Match the repo's existing conventions. 4. **Phase D - Verify.** Re-run every check that failed. Output a before/after table. List the [HUMAN] items the owner must do (external accounts you cannot access). Hard rules, no exceptions: - Never emit schema for content the page does not visibly show. - Never keyword-stuff, cloak, or add hidden text. Being genuinely readable is the whole method. - Never invent facts: no fake client names, prices, dates, or claims. [CONTENT] items get proposed drafts marked as drafts. - One URL form for the whole site (www or not, trailing slash or not). Detect the current dominant form and standardize to it - do not switch an established site's form. --- ## SECTION 0 - Rendering: content in raw HTML (blocks everything else) - [ ] `curl -s https://SITE/ | grep ""` returns a hit. Repeat for one page per template type (home, service page, article). - FAIL fix by stack: SPA -> prerender or SSR every route to static HTML at build time (static prerender suffices for marketing sites). SSG (Astro/Hugo/11ty/Next static) -> verify only. Server-rendered CMS -> verify no JS-only content blocks. - Verify with view-source semantics (raw response), never the hydrated DOM. - [ ] No broken internal links or soft-404s (error content served with a 200). Verify: crawl internal hrefs, assert 200s; fetch a nonsense URL, assert a real 404 status. - [ ] Pages respond fast enough to be fetched. AI fetchers time out and give up on slow pages - which registers as "could not read", the same silent failure as a JS shell. Verify: `curl -o /dev/null -s -w '%{time_total}\n' https://SITE/page` on key pages; flag anything over ~3s. - If this fails and cannot be fixed, stop and report - every other item is decoration on an empty box. ## SECTION 1 - Per-route head tags Every page gets its own: - [ ] Unique ``. - [ ] Unique `<meta name="description">`. **Write it as the liftable answer, not a slogan**: the literal sentence that answers the page's buyer question ("[Name] is an AI automation agency that builds custom AI tools for marketing agencies"). Answer engines lift this verbatim. [CONTENT] - [ ] Absolute `<link rel="canonical">`, self-referencing, in the site's one URL form. - [ ] `<meta name="robots" content="max-snippet:-1, max-image-preview:large, max-video-preview:-1">`. - Verify: `curl -s https://SITE/page | grep -E '<title>|canonical|description'` per template type. ## SECTION 1b - Document semantics - [ ] `<html lang="...">` present and correct for the content language. - [ ] Exactly one `<h1>` per page; heading levels in logical order, no skips. Styled-`div` pseudo-headings read as undifferentiated text to a parser. - [ ] Real landmark elements - `<nav>`, `<main>`, `<article>`, `<header>`, `<footer>` - not styled `<div>`s. - [ ] Every meaningful content `<img>` has descriptive `alt`; decorative images get empty `alt=""`. AI systems read alt text as a content signal; accessibility baseline besides. - Verify: parse each template type - assert one h1, no level skips, no content `<img>` missing an `alt` attribute. ## SECTION 2 - Open Graph / social cards - [ ] `og:title`, `og:description`, `og:image` per page. - [ ] og:image is 1200x630, **served from the site's own domain** (external CDN images fail on some scrapers), with `og:image:width`, `og:image:height`, `og:image:alt` declared. - [ ] `twitter:card` = `summary_large_image`. - Verify: fetch the page, confirm tags present and image URL returns 200 with correct content-type. - [HUMAN] After deploy: check one URL in an OG debugger and force a re-scrape in LinkedIn Post Inspector - scrapers cache aggressively. ## SECTION 3 - Structured data (JSON-LD) Emit only types matching real, visible content: - [ ] Organization (sitewide): name, url, logo, `sameAs` to real profiles (LinkedIn, GitHub, etc. - only URLs that exist). Strongest when the linked profile links back to the site; one-way `sameAs` is weaker. [HUMAN] add the site URL to any profile that lacks it. - [ ] Person on the About/founder page. - [ ] Article on posts: headline, datePublished, dateModified, author. - [ ] FAQPage on any page with a visible FAQ - **generated from the same data that renders the visible questions, never schema-only**. - [ ] Service on offering pages, with `serviceType`. - Verify: parse every emitted block as JSON; confirm each claim is visible on the rendered page. [HUMAN] Spot-check one URL in Google's Rich Results Test. ## SECTION 4 - Machine files At the site root, generated from the route/content source so they cannot drift: - [ ] `/sitemap.xml` - every canonical route, auto-generated at build. - [ ] `/robots.txt` - allows crawling, names the sitemap, and takes an explicit position on AI crawlers. Default: allow. Current agent tokens (re-verify against vendor docs, these change): live-fetch `ChatGPT-User`, `Claude-User`, `Perplexity-User`; search index `OAI-SearchBot`, `Claude-SearchBot`, `PerplexityBot`; training/control `GPTBot`, `ClaudeBot`, `CCBot`, `Google-Extended`, `Applebot-Extended`. Do not list deprecated `Claude-Web` / `anthropic-ai`. - [ ] `/llms.txt` - plain-text/markdown description written FOR AI assistants: one paragraph on what the business does, offerings, key pages with one-line descriptions, contact. Generated from the route table. [CONTENT] for the paragraph itself. Honesty note: no major provider has confirmed reading llms.txt - include it because it costs fifteen minutes and the convention may mature, not because it is proven. - Verify: all three URLs return 200; sitemap route count equals canonical route count; robots.txt does not contain `Disallow: /` under `User-agent: *`. ## SECTION 5 - Retrieval targets: content built to be cited, not just read Provenance: the intent-page, dated-Q&A, and liftable-description items (the last lives in Section 1) derive from a measured finding of our monthly recommendation-tracking runs (published at yellowhousedigital.com/writing/ai-visibility-first-measurement): across repeated runs of the same buyer question, one company kept appearing while every other name was a lottery draw, and its site was structured exactly this way. One stable appearance in a high-variance system: treat these as measured hypotheses the instrument keeps testing, not proven rules. The structural layer. Technical readability gets the site seen; these get it quoted. - [ ] **One page per buyer intent.** For each primary buyer question ("who builds X for Y", "how much does Z cost"), a dedicated page whose entire job is answering it. Audit: list the site's target intents, map existing pages to them, report the gaps. Implementing new pages is [CONTENT] - propose page outline + draft, await approval. - [ ] **A visible, dated Q&A block on key pages**: direct questions phrased the way buyers ask, direct answers, a visible "last updated" date. Pair with FAQPage schema from Section 3. - [ ] **Visible dates on articles** (published + updated), matching the Article schema values. - [ ] **One coherent entity story**: hero, footer, about, and offering pages state the same "we do X for Y". Audit and report inconsistencies; rewording is [CONTENT]. - [ ] **Original data where it exists**: if the business produces findings, statistics, or benchmarks, each gets a citable page - a named finding, a number, a date, a stated method. Answer engines quote pages that give them a number to attribute; original data is the strongest citation magnet there is. Audit: report whether such assets exist and whether they have dedicated pages. New pages are [CONTENT]. - [ ] **Answer-first passages**: engines retrieve passages, not pages. Key sections should be self-contained - a question-shaped heading, the answer in the first sentence, support after. Audit key pages for buried answers; rewording is [CONTENT]. - [ ] **No orphan pages**: every important page reachable by a text link from at least one other page - not only from the sitemap. Discovery and citation both weaken for pages nothing links to. Verify: cross-reference sitemap routes against in-body links; report orphans. - [ ] **No gated content**: methods and guides readable without email forms. Gated = uncrawlable = uncitable. Report any gates found. ## SECTION 6 - Indexing endpoints - [ ] **IndexNow**: generate a key file at the root and ping the IndexNow endpoint on deploy. On hosts with native support (many static hosts, some CDNs), enable it. On any other host, a small deploy-time script achieves the same: write a `{key}.txt` file at the site root, then POST `{ host, key, keyLocation, urlList }` to `https://api.indexnow.org/indexnow`, with the URL list read from the sitemap's `<loc>` entries. Run the ping after the new build is live, so the submitted URLs resolve. - No API key or account is needed - the root `{key}.txt` file is the entire ownership proof. (A Bing Webmaster Tools account is a separate, optional thing.) - One ping to `api.indexnow.org` fans out to all participating engines (Bing, Yandex, Seznam, Naver) - do not ping each separately. Google does not participate in IndexNow; Google is covered only by the sitemap plus Search Console, which is why the GSC item below stays separate. - Bing consumes IndexNow - and ChatGPT search has historically leaned on Bing's index, so Bing indexation is worth having regardless of how that plumbing evolves. Recommended on every deploy. - Verify: the key file returns 200 at the root; a manual ping returns 200/202. - [HUMAN] Google Search Console: verify property, submit sitemap, request indexing on key pages. - [HUMAN] Bing Webmaster Tools: same. Can import the GSC property in one click. - The checklist above is the deploy-side minimum. The full account-side method - property setup choices, what to read in the consoles afterwards, and which reports are noise - is at yellowhousedigital.com/writing/telling-the-engines-you-exist ## SECTION 7 - Final sweep - [ ] Re-run Section 0's curl test on every template type, on production. - [ ] Confirm one URL form holds: `curl -I` the three non-canonical variants (http, other-www-form, other-slash-form) - each must 301 to the canonical form in one hop. - [ ] Output the before/after findings table, the list of [CONTENT] drafts awaiting approval, and the [HUMAN] queue. --- ## Report format (Phase B and Phase D) | # | Item | Status before | Status after | Notes | |---|------|---------------|--------------|-------| Plus two lists: `[CONTENT] awaiting approval` and `[HUMAN] owner actions`. ## What this instruction is NOT Not a ranking-hacks list. Anything resembling keyword stuffing, schema-for-invisible-content, or cloaking is a defect, not an optimization. If a check can only pass by faking something, report it as unfixable instead. Not the whole of AI visibility. This sweep makes a site readable and citable - the entry ticket. What it cannot touch: **off-site entity presence** (mentions, reviews, directories, other people's articles - a major driver of who gets recommended) and **the substance of the content itself**. Passing every check here means the site no longer loses on technical grounds; it does not mean the site wins. Report completion in those terms.