主題爬行器的研究與實(shí)現(xiàn).pdf

1、Y1410068分類(lèi)號(hào)UDCtl：1密級(jí)學(xué)位論文主題爬行器的研究與實(shí)現(xiàn)(題名和副題名)薛艷珠(作者姓名)指導(dǎo)教師姓名篷洼塾撞電子科技大學(xué)成都(職務(wù)、職稱(chēng)、學(xué)位、單位名稱(chēng)及地址)申請(qǐng)專(zhuān)業(yè)學(xué)位級(jí)別碩士專(zhuān)業(yè)名稱(chēng)計(jì)算機(jī)應(yīng)用技術(shù)論文提交日期20083論文答辯日期20085學(xué)位授予單位和日期電子科技大學(xué)答辯委員會(huì)主席評(píng)閱人2008年月日注1：注明《國(guó)際十進(jìn)分類(lèi)法UDc》的類(lèi)號(hào)。AbstractAbstractWitlltherapiddevelop

2、mentoftheweb，aseriesofquestionshaveexposedinthetraditionalgenericsearchengine，suchaLslowcoverage，largeresourceoccupancyrate，longtimetoupdatelOWpertinenccandothersTotacklethoseproblemsofgenericsearchengineandtosatisfyspec

3、ificallyUSersubject—orientedquerythetopicfocusedcrawlingtechnologyCalTleintobeing1nhefocusedcrawlingbaSedonthegenericsearchengine’Stechnologyappliesmachinelearningandotherintelligencemethods，todownloadmorerelatedpagewith

4、lowcostTopicfocusedcrawlingtechnologyhasoccupiedapositionwitllitslli曲degreeofspecializationandobjectivesinnextdevelopmentofsearchenginefrom1990s。Currentlytheresearchesoftopicfocusedcrawlingmainlyconcentrateontwohotspots：

5、DocumentCategorationandCrawlingStrategyDocumentCategorationtechnologyisstudiedinthisthesisTopicfocusedcrawlingusuallyclassifydocumentsbythealgorithmofTFIDFofSVMHoweverTFIDFalgorithmonlytakestheterm’sfrequency(耶)andthedoc

6、umentfiequency(DF)intoconsiderationItneglectstheterm’spositionalinformationanddifferentpositionaltermhasdifferentimpotencetoclassifythedocumentTosolvethisproblem，“Aalgorithmofterraweightingbasedoninformationoftermpositio

7、n’’isproposedinthispaper：assigndifferentweightingfactorstodifferentpositionalterm，thereforetheweightingvaluesofthetermCallreflecttheimportanceofthetermobjectivelyThisalgorithmimprovestheveracityofdocumentcategorizaationT

8、heweightingfactorsCanbeadjustedtogetgoodresultAccordingthedisadvantageofBest—Firstsearch，andtheinformationpeopleusedwhentheyjudgewhetherthehyperlinksaleusefultohimornot，“AcrawlingstrategybaSedoncomprehensiveinformationof

9、URL”isproposedinthisthesisItcalculatesthepredictedcorrelationoftheURLbyanalyzingthesimilarityofcurrentWebpage，URLdirectoryinformationandanchortextofhyperlink，thenitputtheURLtodifferentprioritycrawlingqueueaccordingtheURL