The four methods
- sitemap.xml: open yoursite.com/sitemap.xml. Fastest, often 80–95% coverage. Larger sites split into sitemap indexes — follow the links.
- Search engine: search site:yoursite.com. Only shows indexed pages, but requires no access. Good as a cross-check.
- Crawler: let a tool follow all internal links. Finds pages the sitemap misses, and surfaces broken links and redirect chains along the way.
- CMS export: WordPress, Shopify and others can export a complete page list. The most authoritative source — if you have backend access.
Once you have the list: quality checks
A page list is just the beginning. What you're usually hunting for are the problems: duplicate title tags, missing meta descriptions, 404s and redirect chains. That work is called a technical SEO audit — and much of it can be automated:
- Status codes: every page should return 200. 301s and 404s on the list mean broken internal links.
- Title + description: unique, non-empty, roughly 30–60 and 70–160 characters.
- Canonical: does each page point to itself?
- Open Graph: will the page share properly on social media?
Page Profile takes a URL and shows exactly these fields — the Pro version analyses multiple pages as a batch:
Keep reading
Also see: page size and weight, canonical tags and redirect chains.
Frequently asked questions
Is sitemap.xml enough to find every page?
No. A sitemap only contains what the CMS has registered — typically 80–95%. Old pages, landing pages built outside the CMS and pages excluded by plugins are missing. Use the sitemap as a starting point, not as the complete list.
What's the difference between crawling and a sitemap?
A sitemap is self-reported; a crawler follows links and finds pages that are actually reachable. Crawling catches pages the sitemap misses but can overlook pages with no internal links (orphan pages). Combine both for completeness.
Can I use Google to find pages on a domain?
Yes: search site:yoursite.com and page through results, or use Search Console's page index report if you own the site. Google only shows what it has indexed — new or noindexed pages won't appear.
How do I get metadata for all those pages at once?
Once you have the URL list, pull title, description and status code per page with a tool like Page Profile — batch analysis gives you a table of missing meta tags and error codes without opening each page manually.
Related: Page size checker · Canonical tags explained · Redirect chains