SEO On-Page
Technical SEO Basics

Technical SEO Basics

Technical SEO refers to the process of optimizing your website for the crawling and indexing phase. It ensures search engines can access, crawl, interpret, and index your website without any problems.

Key Technical SEO Elements

Crawlability

Can search engines find and access your pages? This involves robots.txt, internal linking, and XML sitemaps.

Indexability

Can search engines add your pages to their index? This involves noindex tags, canonical URLs, and duplicate content.

Robots.txt

The robots.txt file tells search engine crawlers which pages they can or cannot request from your site.

# Example robots.txt
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/

Sitemap: https://example.com/sitemap.xml
Important: Robots.txt is a public file. Don't use it to hide sensitive content - use password protection instead.

XML Sitemap

An XML sitemap lists all your important pages, helping search engines discover your content.

  • Include all important pages you want indexed
  • Exclude noindex pages, redirects, and error pages
  • Submit your sitemap to Google Search Console
  • Reference your sitemap in robots.txt

Canonical URLs

Canonical tags tell search engines which version of a page is the "master" when similar content exists at multiple URLs.

<link rel="canonical" href="https://example.com/page">

Use canonical tags to handle:

  • www vs non-www versions
  • HTTP vs HTTPS versions
  • Trailing slash variations
  • URL parameters (sorting, filtering)

Noindex Tag

Use the noindex tag to prevent specific pages from appearing in search results:

<meta name="robots" content="noindex">

Pages to consider noindexing:

  • Thank you/confirmation pages
  • Search results pages
  • Paginated archive pages (use carefully)
  • Staging/development pages

HTTPS (SSL/TLS)

HTTPS encrypts data between the user and your server. It's a confirmed ranking factor.

https://example.com

Secure, trusted, ranking boost

http://example.com

Not secure, browser warnings, ranking penalty

Hreflang Tags

If you have content in multiple languages or for different regions, use hreflang tags:

<link rel="alternate" hreflang="en" href="https://example.com/page">
<link rel="alternate" hreflang="es" href="https://example.com/es/page">
<link rel="alternate" hreflang="x-default" href="https://example.com/page">

Technical SEO Checklist

  • Use HTTPS across the entire site
  • Create and maintain an XML sitemap
  • Configure robots.txt properly
  • Implement canonical tags for duplicate content
  • Ensure proper 301 redirects for changed URLs
  • Fix broken links (404 errors)
  • Implement structured data where appropriate

External Resources