Настройка Screaming Frog SEO Spider 19.8 для быстрого парсинга

Screaming Frog SEO Spider 19.8

🔎 НАСТРОЙКА SCREAMING FROG SEO SPIDER 19.8 ПО МЕТОДУ ИЗ ВИДЕО ИВАНА ЗИМИНА (https://www.youtube.com/watch?v=JDpx0Kgc9rA)
С НЕКОТОРЫМИ ИЗМЕНЕНИЯМИ, КОТОРЫЕ УКАЗАНЫ НИЖЕ!  

⭐️ 1. Settings > Storage Mode

Database Storage > D:\_ 2025\PROGRAM FILES\Screaming Frog SEO Spider\Storage
Memory Allocation > 6 Gb (возможно от 4 Gb до 14 Gb)

⭐️ 2. Configuration > Spider > Crawl

Page Links > Internal Hyperlinks > ✅ Crawl + ✅ Store
Page Links > External Links > ✅ Crawl + ✅ Store
Crawl Behaviour > ✅ Check Links Outside of Start Folder
Crawl Behaviour > ✅ Crawl Outside of Start Folder 

⭐️ 3. Configuration > Spider > Extraction

Page Details > ✅ Page Title
Page Details > ✅ Meta Description
Page Details > ✅ Meta Keywords
Page Details > ✅ H1
Page Details > ✅ H2
Page Details > ✅ Indexability
Page Details > ✅ Page Size

⭐️ 4. Configuration > Spider

Limit Crowd Total > 10 000 000

⭐️ 5. Configuration > Exclude

http.*\.jpg
http.*\.jpeg
http.*\.png
http.*\.gif
http.*\.pdf
http.*\.JPG
http.*\.JPEG
http.*\.PNG
http.*\.GIF
http.*\.css
http.*\.CSS
http.*\.js
http.*\.JS
http.*\.jpe*
http.*\.JPE*
http.*\.webp
http.*\?.*
http://.*

⭐️ 5. Configuration > Speed

Max Threads > 5
✅ Limit URL/s
Max URL/s > 5

⭐️ 6. Configuration > User-Agent

Preset User-Agents > Googlebot (Desktop)
HTTP Reques User-Agent > Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Robots User-Agent > Googlebot

⭐️ 7. Main Window > Tabs

✅ Internal
✅ External
✅ URL
✅ Page Titles
✅ Meta Description
✅ Meta Keywords
✅ H1
✅ H2

⭐️ 8. Sidebar Right Window > Tabs

✅ Overview
✅ Site Structure

💾 9. Configuration > Profiles > Save Current Configuration as Default