Validity, Reliability and Accuracy November 6, 2012

Validity, Reliability and Accuracy

 

Purpose:

The purpose of this lesson plan is to have students apply analytical skills and knowledge of literary analysis to reading and writing about poetry, specifically by demonstrating their ability to create autobiographical poems. 

Learning Objective:

Students will learn to research, identify with others, and apply information in poetic ways.

Learning Outcome:

Apply analytical skills and knowledge of literary analysis to reading and writing about poetry.

Materials Needed:

· Internet access (optional)

· Pencils

· Paper

· Encyclopedias

· Biographies printed from the Internet

· A selection of autobiographies and biographies from the library

· Construction paper

· Glue

Lesson Plan:

1.      Students will use the format provided to create poems about themselves. 

· ____________ (first name)

· ____, ____, ____ (three personal characteristics or physical traits)

· Brother or sister of _____ (or son or daughter of)

· who loves ______, _____, and _____? (three people, things, or ideas)

·  Who feels _____ about _____ (one emotion about one thing)

·  Who needs ____, _____, and ____(three items people need)

·  Who gives ____, _____, and ____(three objects people share)

·  Who fears ____, _____, and ____(three items)

·  Who would like to see ______(one place or person)

·  Who dreams of _______(one item or idea)

·  A student of _____ (your school or teacher's name)

·  _______(nickname or repeat your first name)

2.      The teacher will explain to the students what biographies and autobiographies are.  The teacher will show the class several biography examples of famous people.  The teacher will explain to the students that they will be researching a historical figure of their choice.

3.       Students will then use the information collected from their research to create a biographical poem about the famous historical person that they chose.  Students will use the same format from earlier to create poems about the historical figure. 

4.      The teacher will model the assignment on the board by writing a biographical example poem using the information provided in one of the biography examples.  The teacher will say each step of the process aloud as he or she writes it.  The teacher will also show the students the grading rubric that will be used when assessing the assignment.  The teacher will review all of the assignments necessary components prior to having the students work independently.

5.      Students will illustrate a picture each poem with a picture of himself or herself or the historical person researched.  Poems can then be mounted on construction paper and displayed in the classroom.

Assessment:             

Students will be assessed using the Holistic Rubric below.

Autobiographical Poem
	Poor 0	Fair 1	Good 2	Excellent 3
Format	No Attempt	Does not follow the formats given, and had no previous permission to stray from those formats	Completes part or most of the format given, but not all	Follows the formats given
Comprehension	No Attempt	Shows no comprehension of the characters, lives, or influences	Shows some comprehension of the characters, but comprehension remains mainly surface-based.	Goes deeper into the characters, understands their integrity
Knowledge	No Attempt	Shows little understanding of assignment.	Shows some understanding of assignment.	Understands assignment completely.

Vaidity and Reliability:

Assessments should be both valid and reliable in order to accurately display a snapshot of student understanding.  A test has validity evidence if we can demonstrate that it measures what it says it measures (Kubiszyn & Borich, 2010).  The reliability of a test refers to the consistency with which it yields the same rank for individuals who take the test more than once (Kubiszyn & Borich, 2010).  All tests are subject to various sources of error that impair the reliability of their scores and, consequently, their accuracy (Kubiszyn & Borich, 2010).  There are several ways that validity and reliability will be promoted in this assessment. 

Assessments must be created for specific learning objectives.  A test has validity evidence if we can demonstrate that it measures what it says it measures (Kubiszyn & Borich, 2010).  Validity is the degree to which an assessment actually measures the objectives set forth in the lesson plan.  There are three types of validity in regards to classroom testing situations; content, criterion-related and construct.  Content validity is established by inspecting test questions to see whether they correspond to what the user decides should be covered by the test (Kubiszyn & Borich, 2010).  I will promote validity in this holistic rubric by inspecting each area of the rubric to ensure that they correspond to the learning objectives stated at the beginning of the lesson.  Criterion-related evidence is demonstrated by the ability of the test to relate to external requirements (Kubiszyn & Borich, 2010).  Construct validity evidenced takes into account relationships to other information (Kubiszyn & Borich, 2010).  I will continue to promote validity in this assessment by making sure that lesson goals and objectives are clearly defined, and that students are aware of the assignment expectations.  Expectations will be reviewed, written, and clearly displayed while teacher is modeling the assignment.  The teacher will also get the students involved when reviewing the assessment rubric.  Students will be asked to look for problematic language or other difficulties with the proposed rubric.

Reliability refers to the extent to which assessments are consistent.  A reliable test will yield stable scores over repeated administrations, assuming the trait being measured has not changed (Kubiszyn & Borich, 2010).  Classroom tests require three main reliability measures: stability, alternate form and internal consistency (Kubiszyn & Borich, 2010).  The reliability of the assessment can be determined through the use of the test-retest method.  This rubric can be administered twice and the correlation between the first set of scores and the second set of scores can be determined. 

No test is perfectly reliable, and therefore no test is perfectly accurate (Kubiszyn & Borich, 2010).  All tests are subject to various sources of error that impair the reliability of their scores and consequently, their accuracy (Kubiszyn & Borich, 2010).  The sources of error can be classified into four categories; test-takers, the test itself, test administration, and test scoring. 

Students, test-takers, are not always consistent and introduce error into the testing process.  Changes in such things as student's attitudes, health, and sleep may affect the quality of their efforts and thus their test-taking consistency. Test takers may make careless errors, misinterpret test instructions, forget test instructions, inadvertently omit test sections, or misread test items.  As an educator we can address this source of error by modeling what is expected of test-takers, have expectations clearly visible, and reviewing the test rubric prior to the completion of the assignment.

The test itself can be a source of error.  A poorly designed test can result in an obtained score lower than a true score (Kubiszyn & Borich, 2010). This includes the effectiveness of the distractors, wrong options, in multiple choice tests, partially correct distractors, more than one correct answer, poorly written items, and items that are too difficult in relation to student ability.  I will have an outside source, in addition to myself, review this assessment prior to administering it to check for errors within the test. 

There are a variety of test administration factors that can potentially influence the outcome of an assessment.  Error within the test administration refers to physical, verbal, and attitudinal variables that vary from administration to administration and affect test scores (Kubiszyn & Borich, 2010).  The administrator can minimize this source of error by standardizing administration procedures (Kubiszyn & Borich, 2010).  Rubrics are a tool to promote standardization.  This assessment tool can potentially minimize this source of error.

Scoring errors are another potential source of errors. Sources of error include clarity of the scoring rubrics, clarity of what is expected of the student, and test administrator errors. Test administrators are not always consistent, sometimes change their criteria while scoring, and are subject to biases.  This issue can be addressed by clearly defining student expectations, having an outside source such as a fellow teacher or supervisor review the rubric prior to administration to check or biases. 

Assessments that are both valid and reliable can have positive results in the classroom.  As educators, it is our responsibility to assess student learning.  There are many ways to promote validity and reliability in assessment.  Relating test items to lesson objectives and allowing coworkers to preview teacher created assessment tools will promote validity and reliability in assessments.

Works Cited

Arnold, M. (2012, November 06). A Mirror Into History. Retrieved from Education world: http://www.educationworld.com/a_tsl/archives/01-1/lesson0037.shtml

Kubiszyn, T., & Borich, G. (2010). Educational Testing & Measurement Classroom Application and Practice. Hoboken: John Wiley & Sons, Inc.

Reazon Systems, Inc. (2012, November 06). iRubric: Performance Assessment - Autobiographical Poem rubric. Retrieved from Rcampus: http://www.rcampus.com/rubricshowc.cfm?code=N6W46A&sp=yes&