INILAH robot.txt Facebook

Spoiler

# Notice: if you would like to crawl Facebook you can
# contact us here: http://www.facebook.com/apps/site_scraping_tos.php
# to apply for white listing. Our general terms are available
# at http://www.facebook.com/apps/site_scraping_tos_terms.php

User-agent: baiduspider
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: Googlebot
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: msnbot
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: naverbot
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: seznambot
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: Slurp
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: teoma
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: Yandex
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /l.php
Disallow: /o.php
Disallow: /p.php
Disallow: /photo.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php

User-agent: *
Disallow: /

# E-mail sitemaps@lists.facebook.com if you are authorized to access these and are getting denied.
Sitemap: http://www.facebook.com/sitemap.php

source: seo.ddl-turkey.net/tools

Di situ terdapat 77 tools SEO yg sgt bermanfaat. Ada juga alexa multi checker dsb..


My Point is...

Spoiler

Nothing.

Spoiler

I just wanna share.

Lalu format terbaik buat robot.txt kayak gimana ya? Yang agar main auto2an jadi aman, bahkan agc pun jadi aman

Spoiler

User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*?*
Disallow: /*?
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.xlsx $
Disallow: /*.doc$
Disallow: /*.pdf$
Disallow: /*.zip$

User-agent: ia_archiver
Disallow: /

User-agent: duggmirror
Disallow: /

User-agent: *
Disallow: /wp-includes
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /trackback/
Disallow: /feed/
Disallow: /*/feed
Disallow: /comments
Disallow: /tag/
Disallow: /author/
Disallow: /category/
Disallow: /archives
Disallow: /wget/
Disallow: /httpd/
Disallow: /i/
Disallow: /f/
Disallow: /t/
Disallow: /c/
Disallow: /j/
Disallow: /2010/*
Disallow: /2011/*
Disallow: /2012/*
Disallow: /iframes
Disallow: /privacy-policy.html
Disallow: /web-site-agreement.html
Disallow: /category/*/*
Disallow: */trackback


User-agent: *
Allow: /images
User-agent: Mediapartners-Google
Allow: /

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Allow: /

User-agent: ia_archiver-web.archive.org
Disallow: /


Sitemap: http://NAMADOMAINMU.com/sitemap.xml
Sitemap: http://NAMADOMAINMU.com/sitemap.xml.gz

TAMBAHAN:

Spoiler

User-agent: *
Disallow: /

User-agent: Googlebot
Allow: /
Disallow: /feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /feed/
Disallow: /trackback/
Disallow: /rss/
Disallow: /comments/
Disallow: /cgi-bin/
Disallow: /*/comment-page
Disallow: /?s=
Disallow: /*%2520
Disallow: /search/
Disallow: /topik/*%2520
Disallow: /topik/*%252520
Disallow: /topik/*+$
Disallow: /topik/*+.html$
Disallow: /*/$
Disallow: /*--
Disallow: /*++
Disallow: /topik/+
Disallow: /topik/*-
Disallow: /topik/*cache
Disallow: /topik/*elvimiradi
Disallow: /search?
Disallow: /*sex
Disallow: /*porn
Disallow: /*gambling
Disallow: /*poker
Disallow: /*seks
Disallow: /*pharm
Disallow: /*pay
Disallow: /*money
Disallow: /*http
Disallow: /*www
Disallow: /*-.html$
Disallow: /*-online
Disallow: /*4u
Disallow: /*adderall
Disallow: /*adipex
Disallow: /*advicer
Disallow: /*ambien
Disallow: /*baccarat
Disallow: /*baccarrat
Disallow: /*blackjack
Disallow: /*bllogspot
Disallow: /*booker
Disallow: /*byob
Disallow: /*car-rental-e-site
Disallow: /*car-rentals-e-site
Disallow: /*carisoprodol
Disallow: /*casino
Disallow: /*casinos
Disallow: /*chatroom
Disallow: /*cialis
Disallow: /*citalopram
Disallow: /*clomid
Disallow: /*coolcoolhu
Disallow: /*coolhu
Disallow: /*credit-card-debt
Disallow: /*credit-report-4u
Disallow: /*cwas
Disallow: /*cyclen
Disallow: /*cyclobenzaprine
Disallow: /*cymbalta
Disallow: /*dating-e-site
Disallow: /*day-trading
Disallow: /*debt-consolidation
Disallow: /*debt-consolidation-consultant
Disallow: /*discreetordering
Disallow: /*doxycycline
Disallow: /*duty-free
Disallow: /*dutyfree
Disallow: /*ephedra
Disallow: /*equityloans
Disallow: /*facial
Disallow: /*femdom
Disallow: /*fetish
Disallow: /*fioricet
Disallow: /*flowers-leading-site
Disallow: /*freenet
Disallow: /*freenet-shopping
Disallow: /*****ing
Disallow: /*gambling
Disallow: /*gambling-
Disallow: /*hair-loss
Disallow: /*health-insurancedeals-4u
Disallow: /*holdem
Disallow: /*holdempoker
Disallow: /*holdemsoftware
Disallow: /*holdemtexasturbowilson
Disallow: /*homeequityloans
Disallow: /*homefinance
Disallow: /*hotel-dealse-site
Disallow: /*hotele-site
Disallow: /*hotelse-site
Disallow: /*hydrocodone
Disallow: /*incest
Disallow: /*insurance-quotesdeals-4u
Disallow: /*insurancedeals-4u
Disallow: /*jrcreations
Disallow: /*levitra
Disallow: /*lexapro
Disallow: /*lipitor
Disallow: /*lorazepam
Disallow: /*macinstruct
Disallow: /*mortgage-4-u
Disallow: /*mortgagequotes
Disallow: /*online-gambling
Disallow: /*onlinegambling-4u
Disallow: /*ottawavalleyag
Disallow: /*ownsthis
Disallow: /*oxycodone
Disallow: /*oxycontin
Disallow: /*palm-texas-holdem-game
Disallow: /*paxil
Disallow: /******
Disallow: /*percocet
Disallow: /*pharmacy
Disallow: /*phentermine
Disallow: /*poker
Disallow: /*poker-chip
Disallow: /*porn
Disallow: /*poze
Disallow: /*propecia
Disallow: /*pussy
Disallow: /*rental-car-e-site
Disallow: /*ringtone
Disallow: /*ringtones
Disallow: /*roulette
Disallow: /*sex
Disallow: /*shemale
Disallow: /*shoes
Disallow: /*slot-machine
Disallow: /*texas holdem
Disallow: /*texas-holdem
Disallow: /*thorcarlson
Disallow: /*top-e-site
Disallow: /*top-site
Disallow: /*tramadol
Disallow: /*trim-spa
Disallow: /*ultram
Disallow: /*valeofglamorganconservatives
Disallow: /*valium
Disallow: /*valtrex
Disallow: /*viagra
Disallow: /*vioxx
Disallow: /*xanax
Disallow: /*zolus
Disallow: /*sexuality-video
Disallow: /*film-blue
Disallow: /*cerita-dewasa
Disallow: /*bokep
Disallow: /******
Disallow: /*******
Disallow: /*toket
Disallow: /*entot
Disallow: /*ngentot
Disallow: /*sepong
Disallow: /*gay
Disallow: /*ngewe
Disallow: /*ngewek
Disallow: /*itil
Disallow: /*bispak
Disallow: /*pelacur
Disallow: /*film-blue
Disallow: /*bersetubuh
Disallow: /*hack
Disallow: /*perkosa
Disallow: /*pemerkosaan
Disallow: /*mesum
Disallow: /*porno
Disallow: /*japanlez
Disallow: /*film-semi
Disallow: /*tvrdica
Disallow: /*janja
Disallow: /*prepricano
Disallow: /*lektira
Disallow: /*sterija
Disallow: /*popovic
Disallow: /*jovan
Disallow: /*miyabi
Disallow: /*Besplatno
Disallow: /*Gledanje
Disallow: /*Zivotinjam
Disallow: /*Jebanje
Disallow: /*hentai
Disallow: /*camfrog
Disallow: /*tante-girang
Disallow: /*bangbros
Disallow: /*gangbang
Disallow: /*orgy
Disallow: /*prostitusi
Disallow: /*pelacuran
Disallow: /*3gp
Disallow: /*seks
Disallow: /*bugil
Disallow: /*baya
Disallow: /*itil
Disallow: /*masturbasi
Disallow: /*onani
Disallow: /*puki
Disallow: /*lucah
Disallow: /*bercinta

User-agent: ia_archiver
Disallow:

User-agent: Slurp
Disallow: /feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /feed/
Disallow: /trackback/
Disallow: /rss/
Disallow: /comments/
Disallow: /cgi-bin/

User-Agent: MediaPartners-Google
Allow: /

Spoiler

Apabila postingan ini bermanfaat silahkan pencet like.
Jika tdk silahkan tekan back pada browser anda.

Spoiler

Saya tak suka berdebat, cukuplah korban peperangan di muka bumi ini.

Spoiler

Lagipula ane kagak takut

 

jempol lgsg melayang gan,

 

ngakak gua baca tulisan yg di garis
thank anyway ,thanks penceted.....

 

sumber :

Code:

http://facebook.com/robots.txt

 

ini apaan y? kok aku tambah bingung

 

hehehee, tertarik ama yang ini nih:

sama kaya punyaku ternyata

 

yg di garis ane kagak ngarti. maklum nubie...

 

Apalagi ane

 

ehem ehem ...

Spoiler

oooooooooohhhhhhhhhhh

Spoiler

beeeeggggggggoooooooonnnnnnnnnnnnnoooooooo

Spoiler

ttooooooooohhhhhhh

Spoiler

 

O Yea?

Halaman yang Anda minta tidak ditemukan

Anda mungkin mengklik tautan yang kedaluwarsa atau salah menuliskan alamat. Sebagian alamat membedakan kapitalisasi.

• Kembali ke Beranda
• Kembali ke halaman sebelumnya

masta yg di apatar sopo to iku? sodara nya ya?

mau dong dikenalin

---------- Post added 01-16-2011 at 12:08 AM ---------- Previous post was 01-15-2011 at 11:56 PM ----------

gimana daeng, udah ada isi na?

Spoiler

---------- Post added at 12:11 AM ---------- Previous post was at 12:08 AM ----------

I like it

Spoiler

your ehem is sexy

 

Lebih bagus lagi ini, punya Google

Spoiler

User-agent: *
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalogues
Disallow: /news
Allow: /news/directory
Disallow: /nwshp
Disallow: /setnewsprefs?
Disallow: /index.html?
Disallow: /?
Disallow: /addurl/image?
Disallow: /pagead/
Disallow: /relpage/
Disallow: /relcontent
Disallow: /imgres
Disallow: /imglanding
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
Disallow: /advanced_group_search
Disallow: /googlesite
Disallow: /preferences
Disallow: /setprefs
Disallow: /swr
Disallow: /url
Disallow: /default
Disallow: /m?
Disallow: /m/?
Disallow: /m/blogs?
Disallow: /m/directions?
Disallow: /m/ig
Disallow: /m/images?
Disallow: /m/imgres?
Disallow: /m/local?
Disallow: /m/movies?
Disallow: /m/news?
Disallow: /m/news/i?
Disallow: /m/place?
Disallow: /m/products?
Disallow: /m/products/
Disallow: /m/setnewsprefs?
Disallow: /m/search?
Disallow: /m/swmloptin?
Disallow: /m/trends
Disallow: /m/video?
Disallow: /wml?
Disallow: /wml/?
Disallow: /wml/search?
Disallow: /xhtml?
Disallow: /xhtml/?
Disallow: /xhtml/search?
Disallow: /xml?
Disallow: /imode?
Disallow: /imode/?
Disallow: /imode/search?
Disallow: /jsky?
Disallow: /jsky/?
Disallow: /jsky/search?
Disallow: /pda?
Disallow: /pda/?
Disallow: /pda/search?
Disallow: /sprint_xhtml
Disallow: /sprint_wml
Disallow: /pqa
Disallow: /palm
Disallow: /gwt/
Disallow: /purchases
Disallow: /hws
Disallow: /bsd?
Disallow: /linux?
Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /answers/search?q=
Disallow: /local?
Disallow: /local_url
Disallow: /froogle?
Disallow: /products?
Disallow: /products/
Disallow: /froogle_
Disallow: /product_
Disallow: /products_
Disallow: /products;
Disallow: /print
Disallow: /books
Disallow: /bkshp?q=
Allow: /booksrightsholders
Disallow: /patents?
Disallow: /patents/
Allow: /patents/about
Disallow: /scholar
Disallow: /complete
Disallow: /s?
Disallow: /sponsoredlinks
Disallow: /videosearch?
Disallow: /videopreview?
Disallow: /videoprograminfo?
Disallow: /maps?
Disallow: /mapstt?
Disallow: /mapslt?
Disallow: /maps/stk/
Disallow: /maps/br?
Disallow: /mapabcpoi?
Disallow: /maphp?
Disallow: /mapprint?
Disallow: /maps/api/js/StaticMapService.GetMapImage?
Disallow: /maps/api/staticmap?
Disallow: /mld?
Disallow: /staticmap?
Disallow: /places/
Allow: /places/$
Disallow: /maps/place
Disallow: /help/maps/streetview/partners/welcome/
Disallow: /lochp?
Disallow: /center
Disallow: /ie?
Disallow: /sms/demo?
Disallow: /katrina?
Disallow: /blogsearch?
Disallow: /blogsearch/
Disallow: /blogsearch_feeds
Disallow: /advanced_blog_search
Disallow: /reader/
Allow: /reader/play
Disallow: /uds/
Disallow: /chart?
Disallow: /transit?
Disallow: /mbd?
Disallow: /extern_js/
Disallow: /calendar/feeds/
Disallow: /calendar/ical/
Disallow: /cl2/feeds/
Disallow: /cl2/ical/
Disallow: /coop/directory
Disallow: /coop/manage
Disallow: /trends?
Disallow: /trends/music?
Disallow: /trends/hottrends?
Disallow: /trends/viz?
Disallow: /notebook/search?
Disallow: /musica
Disallow: /musicad
Disallow: /musicas
Disallow: /musicl
Disallow: /musics
Disallow: /musicsearch
Disallow: /musicsp
Disallow: /musiclp
Disallow: /browsersync
Disallow: /call
Disallow: /archivesearch?
Disallow: /archivesearch/url
Disallow: /archivesearch/advanced_search
Disallow: /base/reportbadoffer
Disallow: /urchin_test/
Disallow: /movies?
Disallow: /codesearch?
Disallow: /codesearch/feeds/search?
Disallow: /wapsearch?
Disallow: /safebrowsing
Allow: /safebrowsing/diagnostic
Allow: /safebrowsing/report_error/
Allow: /safebrowsing/report_phish/
Disallow: /reviews/search?
Disallow: /orkut/albums
Allow: /jsapi
Disallow: /views?
Disallow: /c/
Disallow: /cbk
Disallow: /recharge/dashboard/car
Disallow: /recharge/dashboard/static/
Disallow: /translate_a/
Disallow: /translate_c
Disallow: /translate_f
Disallow: /translate_static/
Disallow: /translate_suggestion
Disallow: /profiles/me
Allow: /profiles
Disallow: /s2/profiles/me
Allow: /s2/profiles
Allow: /s2/photos
Allow: /s2/static
Disallow: /s2
Disallow: /transconsole/portal/
Disallow: /gcc/
Disallow: /aclk
Disallow: /cse?
Disallow: /cse/home
Disallow: /cse/panel
Disallow: /cse/manage
Disallow: /tbproxy/
Disallow: /imesync/
Disallow: /shenghuo/search?
Disallow: /support/forum/search?
Disallow: /reviews/polls/
Disallow: /hosted/images/
Disallow: /ppob/?
Disallow: /ppob?
Disallow: /ig/add?
Disallow: /adwordsresellers
Disallow: /accounts/o8
Allow: /accounts/o8/id
Disallow: /topicsearch?q=
Disallow: /xfx7/
Disallow: /squared/api
Disallow: /squared/search
Disallow: /squared/table
Disallow: /toolkit/
Allow: /toolkit/*.html
Disallow: /globalmarketfinder/
Allow: /globalmarketfinder/*.html
Disallow: /qnasearch?
Disallow: /errors/
Disallow: /app/updates
Disallow: /sidewiki/entry/
Disallow: /quality_form?
Disallow: /labs/popgadget/search
Disallow: /buzz/post
Disallow: /compressiontest/
Disallow: /analytics/reporting/
Disallow: /analytics/admin/
Disallow: /analytics/web/
Disallow: /analytics/feeds/
Disallow: /analytics/settings/
Disallow: /alerts/
Disallow: /ads/preferences/
Allow: /ads/preferences/html/
Allow: /ads/preferences/plugin
Disallow: /phone/compare/?
Allow: /alerts/manage
Sitemap: http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml
Sitemap: http://www.google.com/hostednews/sitemap_index.xml
Sitemap: http://www.google.com/ventures/sitemap_ventures.xml
Sitemap: http://www.google.com/sitemaps_webmasters.xml
Sitemap: http://www.gstatic.com/trends/websites/sitemaps/sitemapindex.xml
Sitemap: http://www.gstatic.com/dictionary/static/sitemaps/sitemap_index.xml

Bisa dilihat di _http://www.google.com/robots.txt

 

Aduuh setelah ane cermati malah tmbh bingung...hehe nubie nie..