ECU – Post-entry language assessment
Moving from a Post-Entry English Language Assessment (PELA) to an early low stakes assessment task
Edith Cowan University
Dr Anne Harris: Chair ECU ELP Committee, 2013-2015.
Part 1: Background
2009: Initial trials
- In 2007 – 2008, increased students in the Faculty of Business with English as an additional language (EAL). Increased to over 50% in 2009.
- Late 2008, analysis of semester results revealed that students with EAL were among those most ‘at risk’ and lecturers voiced concerns of students’ lack of English Language Proficiency (ELP)
- Early in 2009, the Dean of the faculty secured a substantial grant from the Vice-Chancellor to ‘address ELP’.
- Faculty executive opted to trial two tests: the computer-based ACER English Language Skills Assessment (ELSA) and an IELTS-style test.
- Project manager employed to conduct the tests.
- IELTS-style test (oral and written) was contextualised for business students, trialled in a 2nd year core unit, and results returned within a week.
- ELSA could not initially be conducted due to IT issues. A computer server was sent to Perth in order for the trial to take place, which significantly delayed results.
- Tests evaluated. The test adapted from IELTs was evaluated as relevant for the group but was costly to run. ELSA was perceived as lacking academic items to measure skills.
- University-wide large-scale project that involved analysis of entry pathways highlighted some potentially weak pathways.
- Decision to run more trials. Two paper-based tests: TOEFL Integrated Writing task (reading taken from their textbook, short rebuttal lecture and comparative writing) and a short written task based on a topic contextualised to the discipline. The Integrated writing task was evaluated as the best tool, but was expensive and lengthy to run. Similar results were obtained from the simple written task thus it was the favoured tool for future trials.
- One online tool was tested: The University of Auckland’s DELNA Screening, comprising vocabulary and speed-reading. DELNA Screening was not contextualised to business and the failure rate of computers was 5%.
2010: Establishing adequate learning support
- Faculty keen to implement PELA in 2010 but there was limited support for students.
- Faculty executive supported the establishment of a faculty-based Academic Skills Centre which was operational by March 2010.
- From the onset, learning advisors sought to work mainly within disciplines.
- No PELAs were implemented during this period of change.
2011: Implementing a PELA across the faculty
- Consensus as to the PELA – a short written task – but no agreement as to the approach. PELA process trial conducted as part of university project: Curriculum 2012 and Beyond.
- In 2011, all enrolling students in Business courses received details of the PELA and requirements linked to its completion during Orientation Week.
- The initial completion rate was low (30%) but reminders and additional sessions increased it to 44%. Very few international students participated they were specifically targeted. By end of semester, 50% completion rate, but only 42% of the international intake completed the task.
- Students could pick up feedback a week later but less than 5% opted to do so.
- Students assessed as showing ‘limited development’ were asked to attend a short meeting with a designated learning advisor but less than half took up the offer.
- Both the process and task were evaluated. The short written task was viewed as valid and satisfying needs. Short written task, simply called ‘Written task’ – adopted as ECU’s PELA.
- Due to limited uptake and lack of follow-up, decision taken as to process: the inclusion of the PELA in core units and to work towards its integration into the formal curriculum. Students to complete the task in class and have papers returned to them within two weeks. The provision of learning support to be directly linked to the PELA through embedding of skills in class, adjunct workshops and more general workshops.
- Given that funding for the trials initially came from the Vice Chancellor, there was strong commitment from senior leadership.
Semester 1 2012: Implementing a PELA across the university
- Curriculum 2012 and Beyond Project Report recommended implementation of a PELA in all undergraduate courses, linked to effective learning support. It was approved by university executive.
- PELA team comprising PELA Coordinator and part-time Administration Officer put in place.
- Course coordinators informed about requirement to include the PELA in a unit but offered little or no background about the task. The process was poorly run and students received results seven weeks later.
- There was support from coordinators in the business faculty but wider support was limited to two coordinators from a separate faculty who had been involved in 2011 trials at their request.
Semester 2 2012: Implementing a new feedback strategy
- A new feedback strategy was crucial to widespread acceptance of the PELA. It included consultation with course coordinators, a detailed ‘Feedback Sheet’ for students returned the week following the task, summaries of common errors which would inform learning support, a new process in assessing papers, and timely feedback to all stakeholders.
- The rapid turnaround of papers and sensuring all stakeholders were informed of results were crucial to increased support for the PELA.
Part 2: Current practice
- PELA has been conducted in postgraduate courses since 2014.
- PELA is now simply one aspect of curriculum design. It is included in a first year core unit and forms part of QA processes. It is run on-campus and in the online mode. Online completion rates have generally been significantly lower than on-campus.
- In 2016, the lack of full completion has been addressed and there are now implications for students who do not complete the PELA. This is still in the trial phase.
- By 2013, three courses have integrated the task into the unit’s initial assessment task. Integrated learning support is linked to the task and the feedback utilised for assessment task that follows.
- Results are provided to all stakeholders every semester. They are used to provide targeted academic and language support, analyse entry pathways, and students receive early feedback on their written communication skills.
Part 3: Issues
- At this stage, the PELA remains a process conducted centrally. While costs are low, it remains vulnerable to changing leadership.
- Ideally, all programs will incorporate the PELA as an early low stakes assessment task.
Part 4: High impact practice
- The PELA is fully integrated into curriculum processes and there is follow up should students fail to complete the task.
- Integrated learning support is included in a number of the ‘PELA’ units.
- Feedback stresses the role of students to take responsibility for the development of their communication skills. This links to ongoing feedback via the ECU ELP Measure, in which students receive feedback on all written work.