User-agent:linkCheck
Disallow:/_private/
User-agent:*
Disallow:/webalizer/
Disallow:/search/
revisit-after:5 days