¡¾Nutch¡¿LinuxÏÂÓ¦ÓÃnutch 1.0 Webǰ¶ËʵÏÖµ¥»ú¼ìË÷
nutchµÄÅÀ³æºÍËÑË÷¿ÉÒÔ˵ÊÇ·ÖÀëµÄÁ½¿é£¬ÅÀ³æ¿ÉÒÔÊÇM/R×÷Òµ£¬µ«ËÑË÷²»ÊÇM/R×÷Òµ¡£ËÑË÷ÓÐÁ½ÖÖ·½Ê½£ºÒ»Êǽ«ÅÀ³æÊý¾Ý(»òÕß³ÆË÷ÒýÊý¾Ý)·ÅÔÚ±¾µØÓ²ÅÌ£¬½øÐÐËÑË÷¡£¶þÊÇÖ±½ÓËÑË÷HDFSÖеÄÅÀ³æÊý¾Ý¡£
ÕâÀï½éÉÜÈçºÎʹÓÃnutch-1.0µÄWEBǰ¶Ë¼ìË÷±¾µØÅÀ³æÊý¾Ý£º
(1)NutchµÄËÑË÷¿ÉÒÔ¶ÀÁ¢ÓÚhadoop¼¯Èº£¬Ö»Òª½«ÅÀ³æÏÂÀ´µÄÊý¾Ýcopyµ½ÈκλúÆ÷£¬ÔÚ´Ë»úÆ÷Éϰ²×°Ò»¸ötomcat£¬²¢ÔËÐÐnutch×Ô´øµÄWEBǰ¶Ë³ÌÐò²¢×öÏàÓ¦ÅäÖ㬾ͿÉʵÏÖËÑË÷¡£
(2)½«Ê¹ÓÃÃüÁîbin/nutch crawl -dir data -depth 3 -topN 5ÅÀ³æÏÂÏÂÀ´µÄÊý¾Ýdata·ÅÔÚ±¾µØÄ³Ä¿Â¼Ï£¨Èç¹ûÊÇ·Ö²¼Ê½ÅÀ³æ£¬¿ÉÒÔʹÓÃÃüÁî" bin/hadoop dfs -copyfromLocal data ±¾µØÄ¿Â¼" ½«ÅÀ³æÊý¾Ýdata¸´ÖƵ½±¾µØÄ¿Â¼£©£¬ÀýÈ罫Éú³ÉµÄdataĿ¼¸´ÖƵ½/home/nutch/nutchinstall/crawltest/Ŀ¼Ï¡££¨°²È«Æð¼û£¬ÇëÈ·±£Ä¿Â¼Â·¾¶ÖÐûÓпոñ£¬Õâ¸ö¿ÉÄÜÓÐÓ°Ï죩¡£
˵Ã÷£º
dataĿ¼ÊÇÅÀ³æÉú³ÉµÄĿ¼£¬ÏÂÃæÓÐÕâЩ×ÓĿ¼£ºcrawldb,index,indexes,linkdb,segments
(3)°²×°tomcat£¬ÇëÈ·±£°²×°Â·¾¶Ã»Óпոñ£¬ÕâºÜÖØÒª£¬ÔÚwindowsÉÏÒòΪÓпոñµ¼ÖÂËÑË÷½á¹ûʼÖÕΪ0.
(4)½«NutchÖ÷Ŀ¼ÏµÄWEBǰ¶Ë³ÌÐònutch-1.0.war¸´ÖƵ½ /usr/program/apache-tomcat-6.0.18/webapps/Ŀ¼ÏÂ(apache°²×°Ä¿Â¼ÊÇ/usr/program/apache-tomcat-6.0.18)
(5)ä¯ÀÀÆ÷ÖÐÊäÈëhttp://localhost:8080/nutch-1.0£¬½«×Ô¶¯½âѹnutch-1.0.war¡£
(6)ÅäÖÃWEBǰ¶Ë³ÌÐòÖеÄnutch-site.xmlÎļþ£¬ÅäÖÃÍê³Éºó±ØÐëÖØÆôtomcat(/usr/program/apache-tomcat-6.0.18/bin/shutdown.sh,È»ºóÔÚstart.sh)¡£
nutch-site.xmlÔÚĿ¼/usr/program/apache-tomcat-6.0.18/webapps/nutch-1.0/WEB-INF/classes/Ï£¬
ÅäÖÃÈçÏ£º
<property>
<name>http.agent.name</name> ²»¿ÉÉÙ£¬·ñÔòÎÞËÑË÷½á¹û
<value>nutch-1.0</value>
<description>HTTP 'User-Agent' request header.</description>
</property>
<property>
<name>http.robots.agents</name>
<value>nutch-1.0,*</value>
<description>The agent strings we'll look for in robots.txt files,
comma-separated, in decreasing order of precedence. You should
put the value of http.agent.name as the first agent name, and keep the
default * at the end of the li
Ïà¹ØÎĵµ£º
Ò»¡¢ÒýÑÔ£º
±¾ÎĵµµÄÄÚÈݴ󲿷ÝÄÚÈݶ¼ÊÇ´ÓÍøÉÏÊÕ¼¯¶øÀ´£¬È»ºóÅäºÏһЩеĽØÍ¼£¨Äں˰汾£ºV2.4.19£©¡£ÔÚÿһÅäÖÃÏîºó»áÓÐÒ»¸öÑ¡ÔñÖ¸ÄϵIJ¿·Ý£¬ÓÃÀ´Ö¸µ¼´ó¼ÒÔõôÑù¸ù¾Ý×Ô¼ºµÄÇé¿öÀ´×öÏàÓ¦µÄÑ¡Ôñ£»»¹ÓÐÔÚÿһ¸ö´óÏîºÍÎĵµµÄ×îºó»áÓÐÒ»¸ö¾Ñé̸£¬ËüÊÇһЩ¸ßÊÖÃÇÔÚÓ¦¶ÔÎÊÌâºÍ´¦ÀíÌØÓÐÓ²¼þʱµÄһЩ¾Ñ飨Õâ¸ö»¹µ ......
#!/bin/sh
/sbin/modprobe ip_conntrack_ftp
CONNECTION_TRACKING="1"
ACCEPT_AUTH="0"
DHCP_CLIENT="0"
IPT="/sbin/iptables" # Location of iptables on your system
INTERNET="eth0" # Internet-connected interface
LO ......
Ubuntu10.04¾µÏñ»á³öÏÖÎÞ·¨Ê¶±ð¼üÅÌÊäÈë¡£ÔڵǼ»ÃæÊ±£¬²»ÄÜÊäÈëÃÜÂ룬ÎÞ·¨Õý³£½øÈëϵͳ¡£ ½â¾ö°ì·¨£º £¨1£© ¿ª»§ÆÁÄ»¼üÅ̹¦ÄÜ ÔÚÉÏͼ¿ª»ú½çÃæÔÚÓÒϽǵ¥»÷СÈËͼ°¸£¬ÔÙµ¥»÷“Universal Access Perferences”£¬³ö»÷“Universal Access Perferences”¶Ô»°¿ò£¬ÔÚ¶Ô»°¿òÖÐÑ¡ÖГUse on-screen keybo ......
http://linux.ctocio.com.cn/437/8831437.shtml
ÎÒÃÇÔÚµ÷ÊÔÓ²¼þ°åʱ£¬¾³£ÐèÒª×ö¶à¸öjffs2µÄ¸ùÎļþϵͳӳÏñ£¬ÓÐʱҲҪ¶Ô±ÈÆäËû;¾¶µÃµ½µÄ¿ÉÓõÄjffs2¸ùÎļþϵ
ͳӳÏñ¡£µ«jffs2µÄÎļþϵͳӳÏñ²»Ïó¹âÅ̵ÄÓ³ÏñÎļþÒ»Ñù¿ÉÒÔͨ¹ýloopÉ豸À´¹ÒÔØ……
ÎÒÃÇÔÚµ÷ÊÔÓ²¼þ°åʱ£¬¾³£ÐèÒª×ö¶à¸öjffs2µÄ¸ùÎļþϵͳ
Ó³Ïñ£¬ ......
ÖÚËùÖÜÖªWindowsϵÄÎļþɾ³ýÖ®ºóÊÇÓпÉÄָܻ´µÄ£¬¶ÔÖØÒªÎļþ£¬ÎªÁËÔö¼Ó»Ö¸´µÄÄѶȣ¬ÎÒÃÇ¿ÉÒÔÓÃÎļþ·ÛË鹤¾ß½øÐÐÎļþ·ÛËé¡£ÎÒ×î³£ÓõÄÊǼ¯³Éµ½×ÊÔ´¹ÜÀíÆ÷ÖеÄEraser¡£ÄÇôÔÚLinuxÏÂÓÐûÓÐÀàËÆµÄÈí¼þÄØ£¿ÎÒÔÚһֻ̨ÓÐUbuntuϵͳµÄÖ÷»úÉÏ£¬°²×°ÁËÒ»¸öÎļþ·ÛË鹤¾ßwipe£¬ÓÐЩÏë·¨£¬¼ÇÏÂÀ´¡£
°²×°£ºapt-get install wipe
wipe ......