Technical SEO Fundamentals
Introduction
Technical SEO ensures search engines can efficiently crawl, index, and render your website. Without a solid technical foundation, even the best content may never reach its ranking potential. This lesson covers essential technical optimizations.
Website Crawlability
Crawlability determines whether search engine bots can access and navigate your pages.
Robots.txt Configuration
The robots.txt file instructs crawlers which pages to access or avoid:
Example robots.txt
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/
Disallow: /api/
Disallow: /*?sort=
Disallow: /*?filter=
Sitemap: https://example.com/sitemap.xml
Common Crawlability Issues
- Blocked resources: CSS/JS files blocked in robots.txt
- Orphan pages: Pages with no internal links pointing to them
- Crawl traps: Infinite URL patterns (calendars, filters)
- Redirect chains: Multiple redirects before reaching final URL
XML Sitemaps
Sitemaps help search engines discover all important pages on your site.
Key Term: XML Sitemap
A file that lists all important URLs on your website, helping search engines understand your site structure and discover new or updated content.
Sitemap Best Practices
- Include only canonical, indexable URLs
- Keep sitemaps under 50,000 URLs or 50MB
- Use sitemap index files for large sites
- Update lastmod dates only when content actually changes
- Submit sitemaps via Google Search Console
URL Structure Optimization
Clean, descriptive URLs improve both user experience and SEO:
URL Structure Comparison
| Bad URL | Good URL |
|---|---|
| example.com/p?id=123 | example.com/seo-basics-guide |
| example.com/cat/sub/2024/post | example.com/blog/post-title |
| example.com/PRODUCTS/Blue-Widget | example.com/products/blue-widget |
URL Best Practices
- Use lowercase letters only
- Separate words with hyphens, not underscores
- Keep URLs short and descriptive
- Include target keyword when natural
- Avoid unnecessary parameters and session IDs
Canonical Tags
Canonical tags tell search engines which version of a page is the "master" copy:
Canonical Tag Implementation
<!-- On page: example.com/products/widget?color=blue -->
<link rel="canonical" href="https://example.com/products/widget" />
<!-- Self-referencing canonical (recommended) -->
<link rel="canonical" href="https://example.com/current-page" />
When to Use Canonicals
- Parameter variations (sorting, filtering, tracking codes)
- HTTP vs HTTPS versions
- WWW vs non-WWW versions
- Syndicated or republished content
- Mobile URLs (if separate from desktop)
Site Architecture
A logical site structure helps both users and search engines navigate your content:
Flat vs Deep Architecture
- Flat: Important pages within 3 clicks from homepage (recommended)
- Deep: Many levels of navigation, pages buried 5+ clicks deep
Internal Linking Strategy
- Link from high-authority pages to important target pages
- Use descriptive anchor text (not "click here")
- Create content hubs linking related articles
- Ensure every page has at least one internal link
Practical Exercise
Hands-On Task: Technical SEO Audit
- Check your robots.txt file for blocking issues
- Verify your XML sitemap is accessible and submitted
- Audit 10 URLs for proper structure and canonicals
- Identify pages more than 3 clicks from homepage
- Find and fix any orphan pages without internal links
Summary
- Robots.txt controls crawler access to your site
- XML sitemaps help search engines discover all important pages
- Clean URL structures improve user experience and rankings
- Canonical tags prevent duplicate content issues
- Flat site architecture ensures pages are easily discoverable