语言测试学资料 2.docx
- 文档编号:7537632
- 上传时间:2023-01-24
- 格式:DOCX
- 页数:6
- 大小:18.48KB
语言测试学资料 2.docx
《语言测试学资料 2.docx》由会员分享,可在线阅读,更多相关《语言测试学资料 2.docx(6页珍藏版)》请在冰豆网上搜索。
语言测试学资料2
Chapter2
(第二章)
TheValidityofLanguageTesting
(语言测试的效度)
Whatisvalidity?
Atestissaidtobevalidifitmeasuresaccuratelywhatitisintendedtomeasure.
Validityhasanumberofaspects:
•Contentvalidity内容效度
•Criterion-relatedvalidity标准相关效度
•Constructvalidity编制效度
•Facevalidity表面效度
•Theuseofvalidity效度的用途
ContentValidity:
Atestissaidtohavecontentvalidityifitscontentconstitutesarepresentativesampleofthelanguageskills,structures,etc.withwhichitismeanttobeconcerned.
e.g.
Agrammartestmustbemadeupofitemstestingknowledgeorcontrolofgrammar.
Butthisinitselfdoesnotensurecontentvalidity.Thetestwouldhavecontentvalidityonlyifitincludedapropersampleoftherelevantstructures.
Whataretherelevantstructureswilldependuponthepurposeofthetest.
Inordertojudgewhetherornotatesthascontentvalidity,weneedaspecificationoftheskillsorstructuresetc.thatitismeanttocover.
Suchaspecificationshouldbemadeataveryearlystageintestconstruction.
Itisn’ttobeexpectedthateverythinginthespecificationwillalwaysappearinthetest;theremaysimplybetoomanythingsforallofthemtoappearinssingletest.
Butitwillprovidethetestconstructorwiththebasisformakingaprincipledselectionofelementsforinclusioninthetest.
Acomparisonoftestspecificationandtestcontentisthebasisforjudgmentsastocontentvalidity.Ideallythesejudgmentsshouldbemadebypeoplewhoarefamiliarwithlanguageteachingandtestingbutwhoarenotdirectlyconcernedwiththeproductionofthetestinquestion.
Whatistheimportanceofcontentvalidity?
First,thegreateratest’scontentvalidity,themorelikelyitistobeanaccuratemeasureofwhatitissupposedtomeasure.
Atestinwhichmajorareasidentifiedinthespecificationareunder-represented-----ornotrepresentedatall-----isunlikelytobeaccurate.
Secondly,suchatestislikelytohaveaharmfulbackwasheffect.Areaswhicharenottestedarelikelytobecomeareasignoredinteachingandlearning.Toooftenthecontentoftestsisdeterminedbywhatiseasytotestratherthanwhatisimportanttotest.
Thetestsafeguardagainstthisistowritefulltestspecificationsandtoensurethatthetestcontentisafairreflectionofthese.
Discussion
Case1
Doyouthinkanachievementtestforintermediatelearnerstocontainjustthesamesetofstructuresasoneforadvancedlearnershascontentvalidity?
No.
Case2
About20yearsago,thecandidatesofuniversityentranceexaminationinAmericawasgivenacompositiontopic:
Isphotographyanartorscience?
Discuss.
Doyouthinkthistesthasvalidity?
No.
Case3
Theintentionofotherpeopleconcerned,suchastheMinisterofDefense,toinfluencethegovernmentleaderstoadapttheirpolicytofitinwiththedemandsoftherightwing,cannotbeignored.
Whatisthesubjectof“cannotbeignored”?
A.theintention
B.otherpeopleconcerned
C.theMinisterofDefense
D.thedemandsoftherightwing
Whatdoesthisitemwanttomeasure,readingcomprehensionorsentencestructure?
Criterion-relatedvalidity:
Anotherapproachtotestvalidityistoseehowfarresultsonthetestagreewiththoseprovidedbysomeindependentandhighlydependableassessmentofthecandidate’sability.Thisindependentassessmentisthusthecriterionmeasureagainstwhichthetestisvalidated.
Thereareessentiallytwokindsofcriterion-relatedvalidity:
concurrentvalidity(共时效度)andpredictivevalidity(预时效度).
Whatisconcurrentvalidity?
Concurrentvalidityisestablishedwhenthetestandthecriterionareadministeredataboutthesametime.
e.g.
Courseobjectivescallforanoralcomponentaspartofthefinalachievementtest.
Theobjectivesmaylistalargenumberof‘functions’whichstudentsareexpectedtoperformorally,totestallofwhichmighttake45minutesforeachstudent.Thiscouldwellbeimpractical.
Perhapsitisfeltthatonlytenminutescanbedevotedtoeachstudentfortheoralcomponent.
Thequestionthenarises:
Cansuchaten-minutesessiongiveasufficientlyaccurateestimateofthestudentsabilitywithrespecttothefunctionsspecifiedinthecourseobjectives?
Isitavalidmeasure?
Fromthepointofviewofcontentvalidity,thiswilldependonhowmanyofthefunctionsaretestedinthecomponent,andhowrepresentativetheyareofthecompletesetoffunctionsincludedintheobjectives.
Everyeffortshouldbemadewhendesigningtheoralcomponenttogiveitcontentvalidity.Oncethishasbeendone,however,wecangofurther.Wecanattempttoestablishtheconcurrentvalidityofthecomponent.
Howtodoit?
Weshouldchooseatrandomasampleofallthestudentstakingthetest.
Thesestudentswouldthenbesubjectedtothefull45minuteoralcomponentnecessaryforcoverageofallthefunctions,usingperhapsfourscorerstoensurereliablescoring.
Thiswouldbethecriteriontestagainstwhichtheshortertestwouldbejudged.
Thestudents’scoresonthefulltestwouldbecomparedwiththeonestheyobtainedontheten-minutesession,whichwouldhavebeenconductedandscoredintheusualway,withoutknowledgeoftheirperformanceonthelongerversion.
Ifthecomparisonbetweenthetwosetsofscoresrevealsahighlevelofagreement,thentheshorterversionoforalcomponentmaybeconsideredvalid,inasmuchasitgivesresultssimilartothoseobtainedwiththelongerversion.
If,ontheotherhand,thetwosetsofscoresshowlittleagreement,theshorterversioncannotbeconsideredvalid;itcannotbeusedasadependablemeasureofachievementwithrespecttothefunctionsspecifiedintheobjectives.
Ofcourse,iftenminutesreallyisallthatcanbesparedforeachstudent,thentheoralcomponentmaybeincludedforthecontributionthatitmakestotheassessmentofstudents’overallachievementandforitsbackwasheffect.Butitcannotberegardedasanaccuratemeasureinitself.
‘ahighlevelofagreement’
‘littleagreement’
Howisthelevelofagreementmeasured?
Standardproceduresforcomparingsetsofscores:
‘validitycoefficient’
amathematicalmeasureofsimilarity
Perfectagreementbetweentwosetsofscoreswillresultinavaliditycoefficientof1.
Totallackofagreementwillgiveacoefficientofzero.
Itisbesttosquarethatcoefficient.
acoefficientof0.7betweenthetwooraltests
Squared
0.49
convertedtoapercentage,
49percent
Onthebasisofthis,wecansaythatthescoresontheshorttestpredict49percentofthevariationinscoresonthelongertest.
Inbroadterms,thereisalmost50percentagreementbetweenonesetofscoresandtheother.
Acoefficientof0.5wouldsignify25percentagreement;
Acoefficientof0.8wouldindicate64percentagreement.
Itisimportanttonotethata‘levelofagreement’of50percentdoesnotmeanthat50percentofthestudentswouldeachhaveequivalentscoresonthetwoversions.Wearedealingwithanoverallmeasureofagreementthatdoesnotrefertotheindividualscoresofstudents.
Whatispredictivevalidity?
Predictivevalidityconcernsthedegreetowhichatestcanpredictcandidates’futureperformance.
e.g.
Howwellcouldaproficiencytestpredictastudent’sabilitytocopewithagraduatecourseataBritishuniversity?
Thechoiceofcriterionmeasureraisesinterestingissues:
Shouldwerelyonthesubjectiveanduntrainedjudgmentsofsupervisors?
HowhelpfulisittousefinaloutcomeasthecriterionmeasurewhensomanyfactorsotherthanabilityinEnglish(suchassubjectknowledge,intelligence,motivation,healthandhappiness)willhavecontributedtoeveryoutcome?
Whereoutcomeisusedasthecriterionmeasure,avaliditycoefficientofaround0.4(only20percentagreement)isaboutashighasonecanexpect.
Thisispartlybecauseoftheotherfactors,andpartlybecausethosestudentswhoseEnglishthetestpredictedwouldbeinadequatearenotnormallypermittedtotakethecourse,andsothetest’s(possible)accuracyinpredictingproblemsforthosestudentsgoesunrecognized.Asaresult,avaliditycoefficientofthisorderisgenerallyregardedassatisfactory.
e.g.
Tovalidateaplacementtest:
Placementtestsattempttopredictthemostappropriateclassforanyparticularstudent.Validationwouldinvolveanenquiry,oncecourseswereunderway,intotheproportionofstudentswhowerethoughttobemisplaced.Itwouldthenbeamatterofcomparingthenumberofmisplacements(andtheireffectonteachingandlearning)withthecostofdevelopingandadministeringatestwhichwouldplacestudentsmoreaccurately.
Whatcriterionmeasureshouldwechoose?
Shouldwechooseanassessmentofthestudent’sEnglishasperceivedbyhisorhersupervisorattheuniversity,ortheoutcomeofthecourse(pass/failetc.)?
Constructvalidity
Atest,partofatest,oratestingtechniqueissaidtohaveconstructvalidityifitcanbedemonstratedthatitmeasuresjusttheabilitywhichitissupposedtomeasure.
Theword‘construct’referstoanyunderlyingability(ortrait)whichishypothesizedinatheoryoflanguageability.
Onemighthypothesize,forexample,thattheabilitytoreadinvolvesanumberofsub-abilities,suchastheabilitytoguessthemeaningofunknownwordsfromthecontextinwhichtheyaremet.
Itwouldbeamatterofempiricalresearchtoestablishwhetherornotsuchadistinctabilityexistedandcouldbemeasured.Ifweattemptedtomeasurethatabilityinaparticulartest,
Itwouldbeamatterofempiricalresearchtoestablishwhetherornotsuchadistinctabilityexistedandcouldbemeasured.Ifweattemptedtomeasurethatabilityinaparticulartes
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 语言测试学资料 语言 测试 资料
![提示](https://static.bdocx.com/images/bang_tan.gif)