Homework answers / question archive / Subject Outline Assessment 1: Essay - Evaluation of test or examination Marking Criteria • Demonstrate a clear understanding of concepts relevant to evaluating testing resources • Able to synthesize current discussion in research literature to support critical analysis and discussion of evaluation • Develop and apply appropriate evaluation criteria • Clear and coherent academic writing • Use correctly appropriate referencing styles Length 2,500 words Weighting 40% Assessment Due  19 August 2022 (Sunday in Session Week 6) Final submission time: 11:55pm   Type of Collaboration Individual assessment  Style and format Essay; APA 7th edition Assessment submission

Subject Outline Assessment 1: Essay - Evaluation of test or examination Marking Criteria • Demonstrate a clear understanding of concepts relevant to evaluating testing resources • Able to synthesize current discussion in research literature to support critical analysis and discussion of evaluation • Develop and apply appropriate evaluation criteria • Clear and coherent academic writing • Use correctly appropriate referencing styles Length 2,500 words Weighting 40% Assessment Due  19 August 2022 (Sunday in Session Week 6) Final submission time: 11:55pm   Type of Collaboration Individual assessment  Style and format Essay; APA 7th edition Assessment submission

English

Share With

---

Subject Outline

Assessment 1: Essay - Evaluation of test or examination

Marking Criteria

• Demonstrate a clear understanding of concepts relevant to evaluating testing resources

• Able to synthesize current discussion in research literature to support critical analysis and discussion of evaluation

• Develop and apply appropriate evaluation criteria

• Clear and coherent academic writing

• Use correctly appropriate referencing styles

Length 2,500 words

Weighting 40% Assessment Due  19 August 2022 (Sunday in Session Week 6) Final submission time: 11:55pm

 

Type of Collaboration Individual assessment

 Style and format Essay; APA 7th edition

Assessment submission. Online via Moodle

This assessment has been set up to be checked by Turnitin, a tool which helps you check whether you have referenced correctly. You can submit your assessment task to Turnitin prior to the due date and Turnitin will give you an originality report. You may then make any changes that may be required and resubmit your final version by the due date.

 Assessment return Within 15 working days of the submission date.

 Detailed information Your task is to evaluate a test or examination. The main aim of this task is to help you become familiar with the considerations and decisions involved in designing a test/exam/assessment instrument for NESB learners and whether these decisions have contributed to the validity, reliability or practicality of the instrument.

 You will draw on the knowledge developed throughout the subject (and your readings) to help you critically analyse and evaluate the test/examination/assessment instrument.

Your essay will need to demonstrate a familiarity with the literature on testing in TESOL and the current issues facing the assessment procedure.

1. Write a critical evaluation of one of the following formal or semi-formal assessment procedures in TESOL. Choose one of the following assessment procedures/instruments:

 • a formal examination (e.g. TOEFL, TWE, IELTS, OPI, others)

• a mainstream exam (e.g. Basic Skills Test, final examination, others)

 • an institutionally-developed assessment procedure (e.g. a placement test, diagnostic test, exit test, others)

 • a teacher-made test for second language learners which is used in your institution

 

 2. You will need to:

(a) describe the nature of the assessment procedure (what context is the test used? Who are the targeted test takers? etc. Be as specific as possible as this will help with the evaluation)

(b) use a set of criteria for evaluating the test

 (c) analyse the procedure in light of the criteria

(d) report on its effectiveness or otherwise

 (e) include the chosen assessment procedure as an appendix

 To make this task manageable and allow you to focus on your critical evaluation of the procedure, you will use the set of criteria detailed by Brown & Abeywickrama (2019).

 GENERAL NOTES

 Some of you will be choosing a well-known standardized test such as IELTS or TOEFL. There are many studies and reports on these available in reliable websites. If you analyse one of these tests you will obviously need to do some background research into articles about IELTS and TOEFL. One of the problems with these articles is that most take a positive view and there is not much critical. You can then go into more depth on specific issues related to IELTS or TOEFL such as focusing on one of the macro skills OR on one or more of the above criteria for test design. Sources of information may also

pose some difficulty, for example IELTS, as they are quite protective of their tests and procedures. You will need to exercise tact in order to get the relevant information you need.

Some are choosing national exams such as the College Entrance Exam in China or the high school test in Japan. Again you should read the background research in journals such as JALT or other national journals.

If you reference an article in Chinese or Japanese, put in the name of the journal article in the original language and add a translation in brackets.

Write the name of the author in English transliteration. Some are choosing smaller more local exams. You will not have much primary source research – that is okay. However, you will be expected to do more analysis and investigation, including referring to related literature on the subject to support your claims or make your point.

Important:

Depth or Breadth - You can choose to go into detail, for example, analysing a reading test or you can focus more on broader contextual issues. Reading as much as you can on the topic(s) is essential; it adds not only to the breadth and depth of your critical evaluation, but more importantly to your knowledge. Essays reflect how much of the course readings their writers have done and understood. You will need to also refer to research (beyond the course notes) on things such as multiple choice, cloze or other techniques (communicative language testing, task-based approaches, or portfolios) if you are analysing these topics.

 

Some guiding questions to help you develop your thoughts, ideas, and knowledge gained from the readings, reflections, and professional experiences, if any:

 • Are the test procedures practical? (Practicality) What in the assessment procedure demonstrate this?

 • Is the test reliable? (Test reliability/Rater reliability)

• Does the procedure demonstrate content validity? (Content validity)

 • Is the procedure face valid and “biased for test”? (Face validity)

• Are the test tasks as authentic as possible? (Authenticity)

 • Does the test offer beneficial washback to the learner? (Washback effect)

Suggested outline:

 • Introduction

• Background on the assessment procedure (language program, target student profile, stakeholders, purpose of assessment procedure, others: try to be as specific as possible)

• Proposed set of criteria to be used in the critical evaluation, and rationale

• Description/analysis of the assessment procedure using proposed set of criteria

 • Summation, value judgment (positive/negative)

 • References

• Appendix (including a copy of test or sample items from the assessment procedure