SQE1 in Welsh pilot

Published 11 January 2024

Background

The Solicitors Qualifying Examination (SQE) is the single robust assessment for all aspiring solicitors in England and Wales. It is designed to assure consistent, high standards for all qualifying solicitors.

England and Wales is a single legal jurisdiction, comprising two nations and two official languages. We support access to legal services in both official languages and we have committed to offering the SQE in Welsh by September 2024. Candidates can already choose to take SQE2 in Welsh, and the final stage will be SQE1 in Welsh.

We ran a pilot in June 2023 to explore the practicalities of running the SQE1 assessment in the Welsh language, including the process for translation. The findings are summarised below.

An SQE2 pilot also took place in September 2021.

The assessments

The SQE consists of SQE1, which tests candidates' functioning legal knowledge and SQE2, which tests candidates' practical legal skills.

SQE1 has two parts: Functioning Legal Knowledge 1 (FLK1) and Functioning Legal Knowledge 2 (FLK2). These take place over two non-consecutive days, and both FLK1 and FLK2 are single best answer multiple-choice tests.

SQE2 assesses practical legal skills. It also has two parts: oral assessments over two consecutive half days and written assessments over three consecutive half days.

Read more on the SQE assessment website.

Aims of the SQE1 Welsh pilot

The aim of the SQE1 Welsh pilot was to test specific aspects of conducting an SQE1 assessment in the Welsh language. Specifically, it aimed to:

1. Test the effectiveness of the question translation process.
2. Obtain candidate feedback on alternative options for the presentation of the questions.
3. Reach a conclusion on the use of an assessment specific glossary of terms and / or the publication of a more general glossary used for translation purposes.
4. Assess what information candidates would find helpful to ensure they are able to make an informed decision whether to take the SQE1 in Welsh.
5. Obtain candidate feedback on the overall experience of taking the SQE1 assessment in Welsh.
6. To measure aspects of equivalence in taking the SQE1 assessment in Welsh and English.

The scope excluded testing the validity of the exam as this had been proven in Kaplan's two previous pilots for SQE1 and SQE2.

Pilot development

The 90 SQE1 sample questions were translated into Welsh and used for the pilot.

Promotion of the pilot was through the SQE website, the SQE Update bulletin and social media channels. Information regarding the pilot was also distributed through stakeholder networks.

Candidates were required to have a certain level of legal experience (either from practice or education) but were not sitting the SQE1 assessment for real, and no results were provided to them. 29 candidates took part in the pilot across five locations in Wales and one location in England.

Feedback was gathered from candidates through a post-pilot survey and two focus groups.

Question translation

Planning for the translation of the questions began in 2021 when members of Kaplan SQE held meetings with a panel of Welsh-speaking solicitors to discuss the challenges that may be involved in the translation of the questions into Welsh. This included the difficulty in finding a single accepted term for certain English legal terms and expressions and certain other nuances of the Welsh language. Conversations were held with other stakeholders to discuss these challenges.

The translation of the questions then relied on four main stages:

1. The assessment questions were initially translated by a Welsh translation company which is a full member of Cymdeithas Cyfieithwyr Cymru (the association of Welsh translators and interpreters) as well as the Association of Translation Companies.
2. The translated questions (along with the English versions) were then divided into subject areas and reviewed by a panel of Welsh-speaking solicitors. This stage aimed to help deal with some of the challenges set out above, specifically with the translation of legal terms.
3. Welsh-speaking solicitors then held meetings to discuss their findings with the Kaplan SQE academic team to agree on any amendments that should be made to the questions. The meetings also considered terms that were particularly difficult to translate.
4. The 90 questions were then reviewed by one Welsh-speaking solicitor to help ensure consistency in the use of terminology across the questions.

Question presentation

Throughout the planning and translation process, thought was given to how the questions might be presented and the possible use of a glossary of terms in the assessment. It was decided that the questions would be presented in two different ways:

1. Particularly challenging terms were underlined and accompanied by an on-screen ‘pop up' assessment-specific glossary.
2. Particularly challenging terms were underlined and accompanied by the English translation in brackets within the question.

Summary of findings

The candidate feedback from the survey and the focus groups was considered in the context of the aims of the pilot.

Effectiveness of the question translation process

There were a number of comments relating to the translation of the questions, specifically regarding the literal and formal use of language and the translation of certain terms within the questions. Candidates also considered that certain additional terms should have been accompanied by the English translation/been included in the glossary.

Alternative options for the presentation of the questions

The candidate feedback confirmed a clear preference for the questions to be presented with particular terms underlined, accompanied by the English translation in brackets. Candidates found that the alternative method of presentation (using an on-screen glossary) both interrupted the flow of reading the questions and took additional time.

Use of an assessment specific glossary of terms and/or a more general glossary used for translation purposes

While candidates did not favour the use of an on-screen assessment-specific glossary, the feedback confirmed that having access to resources to assist candidates in becoming familiar with legal terminology in Welsh would be helpful.

Assess what information candidates would find helpful to make an informed decision whether to take the SQE1 in Welsh

Candidates confirmed that the following would be useful:

• Sample questions translated into Welsh and presented as they would be in the assessment.
• The opportunity to compare the Welsh sample questions with the English sample questions.
• Guidance on the process used in translating the questions.
• Assistance with the becoming familiar with legal terminology in Welsh, including the provision of specific resources.
• Guidance on the fluency of Welsh required.

Feedback on the overall experience of taking the assessment in Welsh

Two-thirds of the candidates were either ‘satisfied' or ‘very satisfied' with their overall experience. However, they also fed back that there were issues with the translation of the questions. A little over a third of the candidates said they would choose to sit the SQE1 in Welsh rather than English. In addressing some of the issues referred to above, we hope that more candidates will feel comfortable taking the SQE1 assessment in Welsh.

Measuring aspects of equivalence in taking the assessment in Welsh and in English

An analysis was undertaken on certain aspects of the assessment to measure the equivalence between the Welsh and English versions. Looking at the word count of the 90 sample questions showed that both versions were very similar in length.

The average question response times for the Welsh pilot questions were compared to the average response times for the four previous English SQE1 assessments. This analysis revealed that the average response time was slightly lower for the Welsh questions than the English questions. However, it is appreciated that a direct comparison is difficult given that the Welsh assessment was run as a pilot.

No issues were reported about question length or timing in the candidate feedback.

There were also a small number of comments from candidates who would like to be able to access an English version of the exam alongside the Welsh version. This is how their undergraduate law exams had been presented.

This point was also considered following the SQE2 Welsh pilot and part of discussions with the Welsh Joint Education Committee. The committee felt that, while commonplace in Level 2 and Level 3 assessments, it would be less so for professional exams.

Presenting questions in such a way may also disadvantage candidates given the additional amount of time it would take to review two versions of the same assessment within the time provided. Showing the English alongside the Welsh version is something that we will continue to explore, but there are no plans currently to present the assessment in this way.

Next steps

Translation process

Kaplan now employs an in-house professional Welsh translator who is a member of the Cymdeithas Cyfieithwyr Cymru. A second in-house Welsh translator is due to start in February 2024.

Work is now underway by them to review and improve the translation process, including the use of translation memory software, in order to improve consistency in the use of the Welsh language. The input from Welsh-speaking solicitors will continue as part of the translation process.

Any terms that are found to be ambiguous in Welsh will be referred to Canolfan Bedwyr - Bangor University's Centre for Welsh Language Services, Research and Technology. Its experts in legal terminology and Welsh language matters would then conduct further research and confirm, or make suggestions as to, the term to be used. Sample questions

The SQE1 sample questions on the SQE website are in the process of being translated using the improved translation process. These will be made available on the new SQE1 in Welsh webpage (see below).

The questions will be presented in the form they will appear in the assessment (terms underlined, accompanied by the English translation in brackets). There will also be an equivalent Pearson Vue test centre demonstration site so that candidates can see what the questions will look like on a test platform.

SQE1 in Welsh webpage

We will launch a webpage dedicated to candidates thinking of taking SQE1 in Welsh in spring 2024. All of the candidate feedback received as part of this pilot will be considered carefully when developing this page.

It will contain the translated sample questions, the Pearson Vue demonstration site and further guidance to help candidates to make an informed decision about whether to take SQE1 in Welsh. This would include access to resources to enable candidates to become familiar with legal terminology in Welsh.