校園bbs輿情數(shù)據(jù)收集與提取系統(tǒng)的設(shè)計與實現(xiàn)

1、華中科技大學(xué)碩士學(xué)位論文校園BBS輿情數(shù)據(jù)收集與提取系統(tǒng)的設(shè)計與實現(xiàn)姓名：陳麗萍申請學(xué)位級別：碩士專業(yè)：通信與信息系統(tǒng)指導(dǎo)教師：黃佳慶20090522IIAbstractWiththerapiddevelopmentofInterwksecurityissuesbecomeincreasinglyprominentmeantimethemonitingofwkpublicopinionistobememeimptant.Fcollege

2、suniversitiestheincreasinglycomplexwkenvironmentnotonlybringsmanybenefitsbutalsoposesagreatchallengetotheideologicaleducationsoasapreventivemeasureitisverynecessarytoestablishacompletemechanismtogetthedevelopmentlawofpub

3、licopinionincampuswk.BulletinBoardSystem(BBS)isthemostactiveplatfmincampuswkhenceresearchonmonitingpublicopinioninitbecomesahotspotinwhichthefirstthingshouldbedoneistocollectthewebpagesextracttherelativemessagesffurthera

4、nalysis.OntheabovementionedconditionthispaperfirststudiesthecollectiontechniqueonwebpagesthatiswkcrawlerthenitproposesadatacollectionextractionsystemfBBSincampuswkbycombingthoseprincipletheacteristicsofBBS.Thesystemusesr

5、egularexpressionsffilteringtheUnifmResourceLocats(URLs)whichcanignetheunnecessaryURLsquickly.OntheotherhitadoptstheBloomFilteralgithmtoremovetheduplicatedURLssothestagespacefthevisitedURLsisreducedeffectively.Tofacilitat

6、ethetemplatedextractionwhichcanincreasetheflexibilitygreatlypagesarestedaccdingtothetraitsoftheirURLs.ThetemplateisinthefmatofExtensibleMarkupLanguage(XML)therulesareimplementedbyusinglabelsequenceindex.Thispaperdividest

7、hedataintothreecategieswhicharedownloadedthroughdifferentstrategiesmainlyintroducestheprocessofcollectingupdatedatabyincrementalway.Finallytheresultsofparametertestindicatethatwhenthecycleintervalisabout4hoursthesystemis

8、inthebestconditionwhichmeansithasthelowestrepetitionundetectedrate.Onthisbasisthefunctionaltestresultsrevealthatthesystemcanmeetthedemreachtheintendedtarget.Keywds：wkPublicOpinionBulletinBoardSystem(BBS)WebCrawlerDataCol