外文翻譯--靈活性和計(jì)算機(jī)輔助評價(jià)技術(shù)

1、<p><b>　　中文3582字</b></p><p><b>　　畢業(yè)論文（設(shè)計(jì)）</b></p><p><b>　　外文翻譯</b></p><p><b>　　外文原文</b></p><p>　　FLEXIBILITY AND TH

2、E TECHNOLOGY OF COMPUTERAIDED</p><p>　　ASSESSMENT</p><p>　　R. D. Dowsing</p><p>　　School of Information Systems, University of East Anglia. UK</p><p>　　Email: rdd@sys.u

3、ea.ac.uk</p><p>　　http://www.sys.uea.ac.uk/cats</p><p><b>　　ABSTRACT</b></p><p>　　There are many different facets to flexibility in computer-aided assessment, depending

4、on one’s viewpoint. As an example, for the developer increasing the flexibility of an assessment product means increasing the development cost but also increasing the size of the potential market. For the examiner, flexi

5、bility means the ability to use assessment aids in the specific way and for the specific purpose which he/she requires. For the candidate, flexibility means being given a range of ways to answe</p><p>　　The

6、higher the level of knowledge/skill to be assessed, the more difficult the assessment, the more flexible the assessor needs to be and the greater the involvement of human assessors in the assessment process. For simple t

7、ypes of assessment, for example, the use of multiple-choice questions, the assessment process can be almost completely automated and little human examiner involvement is required. For more complex assessment, for example

8、, assessment of a design rather than an implementation, </p><p>　　Most of the current computerised assessors assess outcome rather than method since this is easier to automate. Techniques are now being devel

9、oped which allow the method used to generate the answer to be collected and assessed. This will give the examiner additional flexibility in the assessment since learners can be assessed by outcome but professionals can b

10、e assessed by method as well as outcome. For example, in assessing IT skills a university student may be assessed for the ability to word p</p><p>　　This paper describes the technology required to add flexib

11、ility to computer-based assessors, with examples, and shows how adding flexibility to an assessor expands the potential uses.</p><p><b>　　KEY WORDS</b></p><p>　　Computerised assessme

12、nt, IT skills, skills assessment.</p><p>　　1. INTRODUCTION</p><p>　　There are three stakeholders in computerised assessment; the system developer, the examiner and the candidates. Each stakeho

13、lder has their own requirements of the assessment system and these requirements can conflict. For the developer, the least risk strategy is to build a system with as much flexibility as possible so that it can then be ta

14、ilored for specific uses by different examiners. Such a system will be applicable to a wide market and thus allow development costs to be amortised over a l</p><p>　　Computerised assessors are changing rapid

15、ly, partly due to technology improvements, partly due to improvements in algorithms and partly due to the increased use and market for such assessors. Presently available assessors are primitive and there is likely to be

16、 rapid development of more flexible and sophisticated systems over the next few years. Developers must possess the ability to predict what future developments might be and build the required flexibility into current prod

17、ucts to enable new </p><p>　　There are two different types of flexibility that a developer has to consider when developing computerised assessors; flexible delivery and flexible use. Flexible delivery implie

18、s that assessment software should be able to be tailored to a specific environment, for example, to run on a selection of hardware or to offer a selection of tests. It also implies that the results of tests and exercises

19、 can be integrated into the user’s present mark processing and recording system. Flexible use implies </p><p>　　The paper concentrates specifically on the assessment of computer-based IT skills from the deve

20、loper’s viewpoint and illustrates the inclusion of flexibility in the development of such assessors with examples from the author’s experience.</p><p>　　2. GENERAL MODELS OF LEARNING AND ASSESSMENT</p>

21、<p>　　There are many different models of learning which have been developed over the years, some of which are discussed in Domjan (1998). A good summary of many of the well-known models can be found in Kearsley (1

22、998). The model which many authors cite in the context of computer assisted assessment is Bloom’s taxonomy, Bloom (1956). He and his committee defined a hierarchical model of learning and assessment where higher levels o

23、f the model relate to higher skills.</p><p>　　Level 1 Knowledge</p><p>　　The ability to remember and recall previously memorized information, for example, to know facts, methods, principles,conc

24、epts and procedures.</p><p>　　Level 2 Comprehension</p><p>　　The ability to grasp the meaning of material, for example, by summarising material or by predicting future trends. This level involve

25、s such processes as translation, interpretation and estimation.</p><p>　　Level 3 Application</p><p>　　The ability to apply knowledge and basic understanding to new situations using such rules, m

26、ethods and principles as the situation requires.</p><p>　　Level 4 Analysis</p><p>　　The ability to break down material into its component parts, understanding the relationship between each of th

27、e parts.</p><p>　　Level 5 Synthesis</p><p>　　The ability to be able to create a new object from a set of components.This requires planning as well as analysis skills.</p><p>　　Level

28、 6 Evaluation</p><p>　　The ability to judge the value of material based on specific criteria.In the Bloom hierarchy, the higher the level of learning and assessment, the greater the flexibility offered to ca

29、ndidates in tests and exercises and the greater the flexibility required of the assessment system. Thus computerised assessors which assess higher levels of the hierarchy are more complex and more costly to produce. At t

30、he lower levels assessment returns either correct or incorrect and there is little information a</p><p>　　Another major difference between the assessment of higher and lower level skills is the data which is

31、 assessed. Lower level exercises almost always assess the outcome of the exercise, for example, the formula typed into a spreadsheet cell as part of a spreadsheet exercise. Higher level skills can also be assessed on the

32、 outcome of an exercise but may also be assessed on the method used by the candidate to generate the outcome, for example, the sequence of key depressions and mouse clicks used. M</p><p>　　3.0 A MODEL OF IT

33、SKILLS ASSESSMENT</p><p>　　In a typical IT skills assessment, as in many other forms of assessment, the exercise/examination follows the following sequence of actions.</p><p>　　1. The examiner p

34、repares the exercise and model answer(s).</p><p>　　2. The candidate sits exam/ does exercise.</p><p>　　3. The candidate’s answer is compared to the model answer(s) to detect raw errors.</p>

35、;<p>　　4. Raw errors are categorised according to the assessment criteria.</p><p>　　5. An assessment is generated from the error analysis.</p><p>　　6. The assessments is recorded for reme

36、dial help/mark generation/student competence tracking</p><p>　　The flexibility in the model is dependent on;</p><p>　　1. The amount of choice the examiner has in setting the exercise.</p>

37、<p>　　2. The amount of choice the candidate has in answering the exercise.</p><p>　　3. The method used to map raw errors to assessment errors.</p><p>　　4. The method of reporting the resul

38、ts.</p><p>　　Each of these is examined in more detail in the following section.</p><p>　　The difficulty in the assessment depends on;</p><p>　　1. The complexity and difficulty of th

39、e exercise set.</p><p>　　2. The number of equivalent correct answers.</p><p>　　3. The complexity of the assessment criteria.</p><p>　　4. The type and amount of feedback required.<

40、;/p><p>　　4.0 FLEXIBILITY IN PRACTICE</p><p>　　There are many different ways in which flexibility is built into practical skills assessors. This section categorises some of the reasons and gives ex

41、amples of such categories of flexibility which have been built into IT skills assessors.</p><p>　　4.1 FLEXIBILITY IN THE MODEL</p><p>　　4.1.1 Flexible Question Setting</p><p>　　In g

42、eneral, many examiners wish to have the ability to set their own exercises, rather than select from the set of exercises provided. Providing the examiner with the tools to set their own exercises can be problematic for t

43、he producer of an automated assessor. The reason for this is that the difficulty of assessing IT skills exercises depends to a large extent on the amount of interaction between individual errors and this interaction betw

44、een errors can be reduced to some extent by careful exer</p><p>　　Example – WordTask Tutor’s Module, Dowsing et al. (1996)</p><p>　　The WordTask word-processing assessor contains a Tutor’s Modul

45、e as part of the suite of assessment programs. This module allows a tutor or examiner to customise the assessor to his/her needs. One of the functions provided allows the tutor to add new exercises to the exercise set as

46、 well as determining the amount of feedback to the student and the amount and type of reporting. This module does not include exercise-vetting checks to warn the tutor of incipient assessment problems although a develo&l

47、t;/p><p>　　4.1.2 Flexible Answers</p><p>　　The result of an IT skills exercise can be assessed either by assessing the outcome, that is, the final document produced, or by assessing the method used

48、 to generate the result. If the final document is assessed then the candidate has the flexibility to use whatever method he/she wishes to generate the correct result. If the method used is assessed then the candidate has

49、 less flexibility. Many skills exercises are assessed by outcome and hence the candidate has the flexibility to use whatever m</p><p>　　Most IT tools provide the user with many equivalent functions to perfor

50、m the same action, for example, there are many different ways of centring a heading using a word processor – centring command, tab, spaces. In some instances it is the appearance of the output of the test on paper which

51、is assessed and thus any of the function combinations which produce the correct appearance are acceptable. In other cases, especially where several people co-operate in the production of a document, the method</p>

52、<p>　　Example – Spreadsheet Assessor based on Event Stream Analysis A spreadsheet assessor has been developed which collects the event stream generated during a spreadsheet exercise. Analysis of the event stream is

53、 performed to determine the sequence of actions the candidate has taken. At present this information is only used to help disambiguate errors in the final output but analysis of the event stream could be performed to ass

54、ess method. This assessor is more complex that the equivalent outcome-ba</p><p><b>　　譯文：</b></p><p>　　靈活性和計(jì)算機(jī)輔助評價(jià)技術(shù)</p><p><b>　　四辛河</b></p><p>　

55、　信息系統(tǒng)學(xué)院，東英吉利大學(xué)。英國</p><p>　　電子郵箱：rdd@sys.uea.ac.uk</p><p>　　http://www.sys.uea.ac.uk/cats</p><p><b>　　摘要</b></p><p>　　在計(jì)算機(jī)輔助評價(jià)中，靈活性隨個人角度不同而具有多面性。例如對于開發(fā)者而言，增加

56、評估產(chǎn)品的靈活性意味著增加開發(fā)成本，但也增加了潛在市場的規(guī)模。對于考官而言，靈活性指的是能夠使用具體方式進(jìn)行輔助評估，達(dá)到他/她要求的特定目的。對于考生而言，靈活性意味著用各種方式來回答一系列問題來充分展現(xiàn)他/她的知識或技能。</p><p>　　評估過程中，評估的知識越多\技巧越高，評估越難，評估者需要更多的靈活性，評估也需要更多的評估人。對于簡單類型的評估，如多項(xiàng)選擇題，評估過程幾乎可以完全自動化，不需要評估

57、人的參與。對于更為復(fù)雜的評價(jià)，如設(shè)計(jì)評價(jià)，而不像實(shí)施評價(jià)那么簡單，測試者有多種可選方案，評估不是簡單依據(jù)對或錯進(jìn)行，而應(yīng)根據(jù)正確性進(jìn)行。計(jì)算機(jī)軟件已成為考官一種輔助用來過濾信息，能夠幫助考官解決部分問題但是問題的最終解決對于考官而言仍然是一個難題。對于自動評估和考官之間的最佳成本效益的平衡，隨時間變化而變化，對于被評估的考生而言而有所不同。 </p><p>　　目前大多數(shù)電腦評審評估只有結(jié)果而看不到方法過程，因

58、為這易于自動化。目前正在開發(fā)的技術(shù)允許用生成答案的方法收集和評估。考官在評估中獲得額外的靈活性，因?yàn)樗纯梢詫W(xué)習(xí)者的結(jié)果進(jìn)行評估，也可以對專業(yè)人員的方法以及結(jié)果進(jìn)行評估。例如，當(dāng)對IT技能的大學(xué)生進(jìn)行評估時，可以從文字處理的能力進(jìn)行對他的評估，當(dāng)評估一個專業(yè)的打字員時，可以根據(jù)他的編輯文章的效率以及正確性來評估。</p><p>　　本文介紹了在以計(jì)算機(jī)為基礎(chǔ)的技術(shù)評估中，如何應(yīng)用技術(shù)來增加靈活性， 以及評估

59、的潛在用途。</p><p><b>　　關(guān)鍵詞</b></p><p>　　電腦化的評估 技能 評估技巧</p><p><b>　　1.0簡介</b></p><p>　　電腦評估中有三個利益相關(guān)者，系統(tǒng)開發(fā)人員，考官和考生。每個利益相關(guān)者都有他們自己評價(jià)體系中的要求，而他們的要求可能也會發(fā)生沖

60、突。對于開發(fā)人員，最重要的戰(zhàn)略是要建立一個系統(tǒng)，使系統(tǒng)盡可能靈活地被用戶用于特定用途。這種系統(tǒng)將適用于一個廣闊的市場，一般的產(chǎn)品比專業(yè)的產(chǎn)品具有更大的銷售范圍。但是，一般用途的產(chǎn)品不像專門針對某一特定的產(chǎn)品好，因此，開發(fā)者具有艱巨的任務(wù)，即平衡開發(fā)的靈活性和開發(fā)成本，來提高產(chǎn)品的適用范圍。</p><p>　　電腦化評估正在迅速變化著，其中部分是由于技術(shù)的進(jìn)步，部分由于算法的改善，以及市場評審使用的增加。目前可用

61、的評審人員都是初級的，在未來數(shù)年里，系統(tǒng)會有較快發(fā)展會更加靈活和復(fù)雜。因此開發(fā)人員必須能預(yù)測未來的發(fā)展可能性，并把必要的靈活性應(yīng)用到當(dāng)前產(chǎn)品中，使新的開發(fā)更加容易和便宜。</p><p>　　現(xiàn)今有兩種不同類型的靈活性，開發(fā)人員開發(fā)時要考慮計(jì)算機(jī)化評估;靈活的交貨方式和靈活運(yùn)用。靈活的交付意味著評估軟件應(yīng)該能夠進(jìn)行調(diào)整以適應(yīng)特定的環(huán)境，例如，對于運(yùn)行時硬件的選擇或提供的測試選擇。這也意味著，其測試的結(jié)果和演習(xí)可以

62、集成到用戶當(dāng)前的標(biāo)記處理和記錄系統(tǒng)。靈活性的運(yùn)用意味著該軟件可以應(yīng)用到不同類型的考試或測試中，例如，形成性測試和總結(jié)性測試。靈活性增加了對評估系統(tǒng)功能的要求，如系統(tǒng)的大小，在一定程度上由評估定制的開發(fā)時間和開發(fā)成本，評估系統(tǒng)要符合大量用戶的具體要求。</p><p>　　本文著重從開發(fā)者的角度介紹了以計(jì)算機(jī)為基礎(chǔ)的信息技術(shù)技能的評估，并根據(jù)評估的發(fā)展和筆者的經(jīng)歷闡明了其靈活性。</p><p&

63、gt;　　2.0學(xué)習(xí)與評估的一般模型</p><p>　　有許多不同的學(xué)習(xí)模型，這些模型是多年發(fā)展而來的。其中一些模型在Domjan（1998年）有討論。在基爾斯利（1998年）中也可以找到一些眾所周知的模型的總結(jié)。許多作者在書中引用的計(jì)算機(jī)評估模型都是Bloom的分類法，Bloom（1956）。他和他的同事定義學(xué)習(xí)和評估的分層模型，他們認(rèn)為在分層模型中模型越高所涉及的技能就越高。</p><

64、p><b>　　級別1.知識</b></p><p>　　記憶和回憶以前記憶信息的能力，例如，要知道事實(shí)，方法，原則，概念和程序。</p><p><b>　　級別2.理解</b></p><p>　　把握材料的含義的能力，例如，總結(jié)材料或通過預(yù)測未來趨勢。這個水平包括筆譯，口譯和估算等過程。 級別3.應(yīng)用&

65、lt;/p><p>　　運(yùn)用知識的能力和在新形勢的情況下使用規(guī)則，方法和原則的能力。</p><p><b>　　級別4.分析</b></p><p>　　能夠把材料分解成幾個組成部分，并了解每個部分之間的關(guān)系的能力。</p><p><b>　　級別5.合成</b></p><p&

66、gt;　　能夠創(chuàng)建的組件設(shè)置一個新的對象的能力,這要求具有規(guī)劃以及分析能力。</p><p><b>　　級別6.評價(jià)</b></p><p>　　根據(jù)具體的標(biāo)準(zhǔn)值判斷材料價(jià)值的能力。</p><p>　　根據(jù)Bloom的級別，測試中提供給考生評估的體系中，越是高的學(xué)習(xí)和評估，越需要更大的靈活性。因此，電腦化評審的評估結(jié)構(gòu)高，級別復(fù)雜，生產(chǎn)成本

67、昂貴。在較低的水平上進(jìn)行評估時，傳回的是正確或不正確，對于他們來講是些沒有什么有意義的反饋信息。例如，用一個選擇題來測試考生是否知道法國的首都哪個。選巴黎的這個答案是明顯正確的，其余的都是不正確的。對于一個不正確的答案，唯一的反饋就是給予正確的回答，可能的話，顯示為什么答案是錯誤的。在較高水平評估中，評估是按正確性度分級的，并有相當(dāng)數(shù)量的有意義的資料反饋給考生。評估者會根據(jù)答案接近正確答案的程度以及所使用的方法來進(jìn)行評估。反饋的信息包括

68、不正確的答案和解釋性的意見。而在較低的水平，精確匹配算法經(jīng)常被使用在高水平下，近似匹配的算法更復(fù)雜和緩慢。它是利用這些近似匹配算法，把高層次的技術(shù)水平的評估從低技能的評估區(qū)分開來。</p><p>　　被評估的數(shù)據(jù)能區(qū)分高度評估和低技能的不同點(diǎn)。低水平的應(yīng)用幾乎總是評估工作的結(jié)果，例如，被輸入到電子表格的公式是作為一個電子表格的工作的一部分。一級技能可以根據(jù)工作的結(jié)果進(jìn)行評估，但也可以根據(jù)考生得出結(jié)果的方法進(jìn)行評

69、估。例如，密鑰序列和鼠標(biāo)的使用。這個方法在團(tuán)體工作技能的測試中尤為重要。</p><p>　　3.0技能評核的模型</p><p>　　一個典型的IT技能評估，就像許多其他形式的評估一樣，練習(xí)/考試都遵循以下順序步驟。</p><p>　　1.考官出考試試題以及標(biāo)準(zhǔn)答案。</p><p>　　2.考生坐在考場考試。</p>&l

70、t;p>　　3.參考標(biāo)準(zhǔn)答案，檢查考生答案的錯誤之處。</p><p>　　4.依照原始的評估標(biāo)準(zhǔn)，對錯誤答案進(jìn)行分類。</p><p>　　5.評估產(chǎn)生誤差分析。</p><p>　　6.把學(xué)生的輔導(dǎo)記錄/ 歷年成績 /才能記錄在評估系統(tǒng)中模型中的靈活性依賴于：</p><p>　　1.考官出的考卷選擇題的數(shù)量。</p>

71、<p>　　2.考生在答題時做題的數(shù)量。</p><p>　　3.把原始錯誤映射到評估錯誤的方法。</p><p>　　4.報(bào)告結(jié)果的方法。</p><p>　　所有這些根據(jù)下面更詳細(xì)部分進(jìn)行考核在評估過程中，遇到的困難取決于以下幾點(diǎn)1.試題的復(fù)雜性和難易程度。</p><p>　　2.正確答案的數(shù)目。</p>

72、<p>　　3.評估的標(biāo)準(zhǔn)的復(fù)雜性。</p><p>　　4.反饋的類型和反饋的數(shù)量。</p><p><b>　　4.0靈活性實(shí)踐</b></p><p>　　在實(shí)用技能的評審中，可以通過許多方式來增加評審的多樣性。通過本節(jié)加以闡述，并列舉了資訊科技技能評估應(yīng)用的靈活性。</p><p><b>　

73、　4.1靈活的模式</b></p><p>　　4.1.1靈活問題的設(shè)置</p><p>　　一般來說，很多考官希望根據(jù)自己的能力出考題，而不是從所提供的練習(xí)集中選取試題。對于自動化評估的生產(chǎn)商來說，給考官提供選題工具是有一定的困難。這是因?yàn)榧寄艿脑u估難度在很大程度上依賴于大量的個人之間誤差的相互作用和錯誤之間的相互作用，但是可以通過精心練習(xí)降低到一定程度。如果測試員或教師根據(jù)

74、自己靈活地考題，那么他們可能會增加評估困難，減少電腦評估的準(zhǔn)確性。但是自動提醒考官不正確的命題的做法可以減少這種可能性。</p><p>　　案例- WordTask模塊，辛等人。（1996）WordTask文字處理評估包含評估方案套件的一部分導(dǎo)師的模塊。這個模塊允許教師或考官自定義他/她的需求評估。該評估所提供的其中一項(xiàng)功能就是允許添加新的導(dǎo)師練習(xí)集以及給學(xué)生反饋報(bào)告的數(shù)量和類型。此模塊不包括行使審閱檢查，以提

75、醒導(dǎo)師的早期評估的問題，但是一些專業(yè)考試中有這些模塊。</p><p>　　4.1.2靈活的答案</p><p>　　IT技能的鍛煉可以通過評估來評估他的結(jié)果，這就是產(chǎn)生的最后文件，或用于生成評估結(jié)果的方法。如果對最終的文件進(jìn)行了評估，那么考官就可以使用任何他/她希望的方法，來生成正確的結(jié)果。如果采用的方法被評估，則是考官所出的試題靈活性小。許多技能都是通過結(jié)果進(jìn)行評估的，因此，考官可以使

76、用任何方法，達(dá)到他/她希望以生成所需的結(jié)果。</p><p>　　大多數(shù)的IT工具提供許多同等功能的用戶來執(zhí)行相同的動作，例如，可以通過使用命令、制表符、空格等文字處理器將標(biāo)題放置成不同方式。在某些情況下，測試評估的輸出形式是紙質(zhì)的，因此任何真確的輸出形式組合都是可以接受的。在其他情況下，特別是有幾個人合作的時候，一起制作文件從而以何種方式產(chǎn)生的效果才是重要的。例如，更改文件中的所有標(biāo)題，使其具有不同屬性和不同的

77、字體風(fēng)格，這樣是簡單，但是如果他們已經(jīng)使用了，但是從來沒有這樣做過，那就困難了。在這種情況下，其實(shí)就是對已經(jīng)產(chǎn)生效果的方法進(jìn)行評估。這包括收集和評估由于受用戶調(diào)用產(chǎn)生的影響的行動順序流事件。收集事件流是比較困難，因?yàn)橐占嘘P(guān)檢測結(jié)果，但它允許通過靈活性和計(jì)算機(jī)技術(shù)輔助評估進(jìn)行額外的分析，以達(dá)到可能的評估標(biāo)準(zhǔn)要求。因此，收集和分析事件流提供了更大的靈活性，但是這不是很普遍，所以需要一個更大的發(fā)展以及努力。</p><