reliablility and vadility.docx
- 文档编号:7253986
- 上传时间:2023-01-22
- 格式:DOCX
- 页数:15
- 大小:80.80KB
reliablility and vadility.docx
《reliablility and vadility.docx》由会员分享,可在线阅读,更多相关《reliablility and vadility.docx(15页珍藏版)》请在冰豆网上搜索。
reliablilityandvadility
Reliability&Validity
Weoftenthinkofreliabilityandvalidityasseparateideasbut,infact,they'rerelatedtoeachother.Here,Iwanttoshowyoutwowaysyoucanthinkabouttheirrelationship.
Oneofmyfavoritemetaphorsfortherelationshipbetweenreliabilityisthatofthetarget.Thinkofthecenterofthetargetastheconceptthatyouaretryingtomeasure.Imaginethatforeachpersonyouaremeasuring,youaretakingashotatthetarget.Ifyoumeasuretheconceptperfectlyforaperson,youarehittingthecenterofthetarget.Ifyoudon't,youaremissingthecenter.Themoreyouareoffforthatperson,thefurtheryouarefromthecenter.
Thefigureaboveshowsfourpossiblesituations.Inthefirstone,youarehittingthetargetconsistently,butyouaremissingthecenterofthetarget.Thatis,youareconsistentlyandsystematicallymeasuringthewrongvalueforallrespondents.Thismeasureisreliable,butnovalid(thatis,it'sconsistentbutwrong).Thesecond,showshitsthatarerandomlyspreadacrossthetarget.Youseldomhitthecenterofthetargetbut,onaverage,youaregettingtherightanswerforthegroup(butnotverywellforindividuals).Inthiscase,yougetavalidgroupestimate,butyouareinconsistent.Here,youcanclearlyseethatreliabilityisdirectlyrelatedtothevariabilityofyourmeasure.Thethirdscenarioshowsacasewhereyourhitsarespreadacrossthetargetandyouareconsistentlymissingthecenter.Yourmeasureinthiscaseisneitherreliablenorvalid.Finally,weseethe"RobinHood"scenario--youconsistentlyhitthecenterofthetarget.Yourmeasureisbothreliableandvalid(IbetyouneverthoughtofRobinHoodinthosetermsbefore).
Anotherwaywecanthinkabouttherelationshipbetweenreliabilityandvalidityisshowninthefigurebelow.Here,wesetupa2x2table.Thecolumnsofthetableindicatewhetheryouaretryingtomeasurethesameordifferentconcepts.Therowsshowwhetheryouareusingthesameordifferentmethodsofmeasurement.Imaginethatwehavetwoconceptswewouldliketomeasure,studentverbalandmathability.Furthermore,imaginethatwecanmeasureeachoftheseintwoways.First,wecanuseawritten,paper-and-pencilexam(verymuchliketheSATorGREexams).Second,wecanaskthestudent'sclassroomteachertogiveusaratingofthestudent'sabilitybasedontheirownclassroomobservation.
Thefirstcellontheupperleftshowsthecomparisonoftheverbalwrittentestscorewiththeverbalwrittentestscore.Buthowcanwecomparethesamemeasurewithitself?
Wecoulddothisbyestimatingthereliabilityofthewrittentestthroughatest-retestcorrelation,parallelforms,oraninternalconsistencymeasure(See TypesofReliability).Whatweareestimatinginthiscellisthereliabilityofthemeasure.
Thecellonthelowerleftshowsacomparisonoftheverbalwrittenmeasurewiththeverbalteacherobservationrating.Becausewearetryingtomeasurethesameconcept,wearelookingatconvergentvalidity(See MeasurementValidityTypes).
Thecellontheupperrightshowsthecomparisonoftheverbalwrittenexamwiththemathwrittenexam.Here,wearecomparingtwodifferentconcepts(verbalversusmath)andsowewouldexpecttherelationshiptobelowerthanacomparisonofthesameconceptwithitself(e.g.,verbalversusverbalormathversusmath).Thus,wearetryingtodiscriminatebetweentwoconceptsandwewouldconsiderthisdiscriminantvalidity.
Finally,wehavethecellonthelowerright.Here,wearecomparingtheverbalwrittenexamwiththemathteacherobservationrating.Likethecellontheupperright,wearealsotryingtocomparetwodifferentconcepts(verbalversusmath)andsothisisadiscriminantvalidityestimate.Buthere,wearealsotryingtocomparetwodifferentmethodsofmeasurement(writtenexamversusteacherobservationrating).So,we'llcallthis very discriminanttoindicatethatwewouldexpecttherelationshipinthiscelltobeevenlowerthanintheoneaboveit.
Thefourcellsincorporatethedifferentvaluesthatweexamineinthe multitrait-multimethod approachtoestimatingconstructvalidity.
Whenwelookatreliabilityandvalidityinthisway,weseethat,ratherthanbeingdistinct,theyactuallyformacontinuum.Ononeendisthesituationwheretheconceptsandmethodsofmeasurementarethesame(reliability)andontheotheristhesituationwhereconceptsandmethodsofmeasurementaredifferent(verydiscriminantvalidity).
IntroductiontoValidity
Validity:
thebestavailableapproximationtothetruthofagivenproposition,inference,orconclusion
Thefirstthingwehavetoaskis:
"validityof what?
"Whenwethinkaboutvalidityinresearch,mostofusthinkaboutresearchcomponents.Wemightsaythatameasureisavalidone,orthatavalidsamplewasdrawn,orthatthedesignhadstrongvalidity.Butallofthosestatementsaretechnicallyincorrect.Measures,samplesanddesignsdon't'have'validity--onlypropositionscanbesaidtobevalid.Technically,weshouldsaythatameasureleadstovalidconclusionsorthatasampleenablesvalidinferences,andsoon.Itisaproposition,inferenceorconclusionthatcan'have'validity.
Wemakelotsofdifferentinferencesorconclusionswhileconductingresearch.Manyofthesearerelatedtotheprocessofdoingresearchandarenotthemajorhypothesesofthestudy.Nevertheless,likethebricksthatgointobuildingawall,theseintermediateprocessandmethodologicalpropositionsprovidethefoundationforthesubstantiveconclusionsthatwewishtoaddress.Forinstance,virtuallyallsocialresearchinvolvesmeasurementorobservation.And,wheneverwemeasureorobserveweareconcernedwithwhetherwearemeasuringwhatweintendtomeasureorwithhowourobservationsareinfluencedbythecircumstancesinwhichtheyaremade.Wereachconclusionsaboutthequalityofourmeasures--conclusionsthatwillplayanimportantroleinaddressingthebroadersubstantiveissuesofourstudy.Whenwetalkaboutthevalidityofresearch,weareoftenreferringtothesetothemanyconclusionswereachaboutthequalityofdifferentpartsofourresearchmethodology.
Wesubdividevalidityintofourtypes.Eachtypeaddressesaspecificmethodologicalquestion.Inordertounderstandthetypesofvalidity,youhavetoknowsomethingabouthowweinvestigatearesearchquestion.Becauseallfourvaliditytypesarereallyonlyoperativewhenstudyingcausalquestions,wewilluseacausalstudytosetthecontext.
Thefigureshowsthattherearereallytworealmsthatareinvolvedinresearch.Thefirst,onthetop,isthelandoftheory.Itiswhatgoesoninsideourheadsasresearchers.Itiswherewekeepourtheoriesabouthowtheworldoperates.Thesecond,onthebottom,isthelandofobservations.Itistherealworldintowhichwetranslateourideas--ourprograms,treatments,measuresandobservations.Whenweconductresearch,wearecontinuallyflittingbackandforthbetweenthesetworealms,betweenwhatwethinkabouttheworldandwhatisgoingoninit.Whenweareinvestigatingacause-effectrelationship,wehaveatheory(implicitorotherwise)ofwhatthecauseis(thecauseconstruct).Forinstance,ifwearetestinganeweducationalprogram,wehaveanideaofwhatitwouldlooklikeideally.Similarly,ontheeffectside,wehaveanideaofwhatweareideallytryingtoaffectandmeasure(theeffectconstruct).Buteachofthese,thecauseandtheeffect,hastobetranslatedintorealthings,intoaprogramortreatmentandameasureorobservationalmethod.Weusetheterm operationalization todescribetheactoftranslatingaconstructintoitsmanifestation.Ineffect,wetakeourideaanddescribeitasaseriesofoperationsorprocedures.Now,insteadofitonlybeinganideainourminds,itbecomesapublicentitythatanyonecanlookatandexamineforthemselves.Itisonething,forinstance,foryoutosaythatyouwouldliketomeasureself-esteem(aconstruct).Butwhenyoushowaten-itempaper-and-pencilself-esteemmeasurethatyoudevelopedforthatpurpose,otherscanlookatitandunderstandmoreclearlywhatyouintendbythetermself-esteem.
Now,backtoexplainingthefourvaliditytypes.Theybuildononeanother,withtwoofthem(conclusion andinternal)referringtothelandofobservationonthebottomofthefigure,oneofthem(construct)emphasizingthelinkagesbetweenthebottomandthetop,andthelast(external)beingprimarilyconcernedabouttherangeofourtheoryonthetop.ImaginethatwewishtoexaminewhetheruseofaWorldWideWeb(WWW)VirtualClassroomimprovesstudentunderstandingofcoursematerial.Assumethatwetookthesetwoconstructs,thecauseconstruct(theWWWsite)andtheeffect(understanding),andoperationalizedthem--turnedthemintorealitiesbyconstructingtheWWWsiteandameasureofknowledgeofthecoursematerial.Herearethefourvaliditytypesandthequestioneachaddresses:
ConclusionValidity:
Inthisstudy,istherea relationshipbetweenthetwovariables?
Inthecontextoftheexamplewe'reconsidering,thequestionmightbeworded:
inthisstudy,istherearelationshipbetweentheWWWsiteandknowledgeofcoursematerial?
Thereareseveralconclusionsorinferenceswemightdrawtoanswersuchaquestion.Wecould,forexample,concludethatthereisarelationship.Wemightconcludethatthereisapositiverelationship.Wemightinferthatthereisnorelationship.Wecanassesstheconclusionvalidityofeachoftheseconclusionsorinferences.
InternalValidity:
Assumingthatthereisarelationshipinthisstudy, istherelationshipa causal one?
Justbecausewefindthatus
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- reliablility and vadility