Home » Posts tagged 'situational judgement'
Tag Archives: situational judgement
A new way to do practice online psychometric tests by law students for legal recruitment
Launched in 2011, ‘Legal Recruit’ is an innovative, totally independent, platform for law students to practice online psychometric tests. These tests are routinely provided by SHL Direct and Kenexa to assist in the recruitment of suitable trainees for training contracts and vacation placements in City law firms. ‘Legal Recruit’ only contains unique, original learning material, not available from any learning provider.
This platform therefore will be ideal for any student who needs to complete a psychometric test for his or her own application. It’s usually expected that a candidate scores in a high percentile, and often firms will not call students who do not meet a minimum standard of performance. On the ‘Legal Recruit’, learners receive an immediate report on their performance, included a detailed breakdown of where they did well on individual items and where they got the answer wrong.
The platform offers students a chance to do 15 full-length verbal reasoning tests and 4 situational judgement tests, for £10. However, there is much free material on the site, including a chance to do a practice online verbal reasoning test for free. 76 graduates have so far taken the test and the average score is 21.61. There are lots of free factsheets, including an introduction to psychometric tests, an introduction to verbal reasoning tests, an introduction to situational judgement tests, visual impairments and reading difficulties, how to write a cover sheet, and how to complete the online application form, and lots of exclusive videos too. We hope to launch a numerical reasoning part of the website shortly.
We were very happy when, last month, the ‘Legal Recruit’ was chosen by the Queen Mary and Westfield College (University of London) Law Students’ Union mock ‘assessment day’, for practice by their law students in verbal reasoning and situational judgement tests. This is excellent co-operation between two active law student groups.
The feedback for ‘Legal Recruit’ has been extremely promising, including from current trainees and law students. There is a free book of 150 verbal reasoning questions that anyone can download for free, also containing original unique specimen test material, and 2 books for comprehensive practice at verbal reasoning and situational judgement tests, priced extremely competitively at £7.5o each.
The link to Legal Recruit is here.
It is important to note that, whilst the website is entirely the intellectual property of ‘Legal Recruit’ (an initiative by students in the BPP Legal Awareness Society run by BPP students for BPP students), this project including product is absolutely nothing o do with BPP. The BPP Legal Awareness Society strives to explain the competitive advantage of businesses through law, and to explain compliance with the law in achieving business strategy.
Online verbal reasoning tests for training contract and vacation scheme placements: introducing Legal Recruit
A few months ago, I got to know the online verbal reasoning test very well. SHL are the one of gold standards in offering the tests, and do so to the best of my constructive knowledge for the vast majority of prestigious corporate law firms here in London. Other test providers, such as Kenexa, are well known, and of course Pearson Assessment are the masterminds behind the Watson-Glaser Critical Thinking Appraisal Test (I know because I have physically sat them in two Head Offices of corporate law firms in the City).
Verbal reasoning tests intrigue me greatly, as my Finals at Cambridge were in neuroscience, and actually I did my Ph.D. there in cognitive neuropsychological assessment. The focus of my Ph.D. was in the skills centred around planning and decision-making, thought to be a function of the frontal lobes, but of course we had to be well versed in other neurocognitive domains such as attention, language, memory and perception. At first I was ambivalent about virtually all the corporate law firms subject future employees to an online verbal reasoning test for about 20 minutes, when you would have thought that a II.1 would be sufficient information for a legal recruiter. Legal recruiters often state that verbal reasoning skills are essential for lawyers, and I used to shrug my shoulders mentally. Actually, I now feel that this is very true. In my Master of Law in commercial law from the College of Law, we were given a lot of practice in drafting various commercial legal documents, such as intellectual property licenses, due diligence reports and even peremptory orders in international arbitration for the High Court. I believe it’s essential for law trainees to be good at basic inferences; they do need to be able to tell whether a statement is true or false from information presented to them, and to be confident enough to say when there is insufficient information to make a conclusion. Knowing the limitations of your own knowledge must be a key skill for any professional.
Therefore, a few months ago, I decided to put my experience of online verbal reasoning tests into action. I am visually impaired, and generally I felt that law firms varied a lot in their basic competence in being allowed to make reasonable adjustments for me, which they are obliged to do under law. For example, on my site http://www.legal-recruit.org, which I launched today, you can also alter the text size, or ask for extra time in doing a test (which learners with dyslexia should be able to action with the help of the Learning Support Officer of their law school and the Graduate Recruitment Team of their chosen corporate). I have done a lot of background reading in the cognitive skills tested in the verbal reasoning test and the situational judgement tests, and reported my findings on my specialist blog http://www.legalrecruit.org). You can even try a practice online verbal reasoning test for fee, and you’ll obtain a free, confidential, detailed report providing you with item-by-item breakdown (as well as your total performance metric.)
On the front page, you can access information about the tests, including factsheets, examples of verbal reasoning tests of the ‘True’, ‘False’ or ‘Cannot Say’ variety, educational videos, and books on verbal reasoning and situational judgment tests.
I have produced factsheets explaining the rationale behind the tests in general, some guidance for the online verbal reasoning test, some guidance about the competences sought by corporate law firms in their situational judgement tests, and what reasonable adjustments you can rightly ask for if you a visual impairment or reading difficulty.
The aim of this website is to make sure that you are familiar with the test format of the online verbal reasoning test. You should of course check the format of the test you need to do with your legal recruiter, and to make sure you understand how the test operates. With help from the books (free sample material is provided on the home page), you can understand how correct answers are arrived at.
I’ve tried to describe this information succinctly in a series of factsheets. However, if you prefer visual material, also for free, we’ve produced some videos for you to understand verbal reasoning tests, situational judgement tests, and reasonable adjustments. As they’re also on YouTube, you can watch these on an iPad.
I hope this makes your journey into the online verbal reasoning test enjoyable! I’ve had great fun working on this venture, which came out of an idea in my student society, but which is otherwise totally independent. Please do let me know how you get on either by commenting here, or emailing enquiries@legal-recruit.org
Psychometric tests for training contract applications
Please note that these videos are from the new Legal Recruit website. Legal Recruit is a project by students in the Legal Awareness Society, a society run by students at BPP for students of law. This Society has nothing to do with BPP in any official capacity, although it is a registered Society at BPP. BPP does not support or endorse any of these independent specific initiatives.
These videos are merely demonstrated for educational purposes, especially to help candidates with visual impairments and reading difficulties to understand the nature of reasonable adjustments in online assessment, and the obligations of the test providers and the law firms ensuring fair assessment. Full details for these videos are provided on the YouTube sites whose links have been provided.
Reading difficulties and visual impairments
http://www.youtube.com/watch?v=luX9qny0T-4&feature=channel_video_titlePsychometric tests
http://www.youtube.com/watch?v=duUwqvVDpMY
What are situational judgement tests?
http://www.youtube.com/watch?v=e7JMlQZbBYk
The situational judgement test: past, present and future
As of this morning, 103 people have taken the BPP Legal Aware Situational Judgement Test.
You can take the test here, if you haven’t done so already.
There has been a small explosion in the research done into ‘situational judgement tests’ (SJT) for employment selection (Weekley and Ployhart, 2006). SJTs present applicants with work-related situations and possible responses to the situations. There are broadly two types of instructions (reviewed by McDaniel et al., 2007). Behavioural tendency instructions ask respondents to identify how they would likely behave in a given situation. Knowledge instructions ask respondents to evaluate the effectiveness of possible responses to a given situation. Tests assessing an individual’s judgement concerning work-related situations have had a long history in the psychological assessment literature (McDaniel et al., 2001). For example, during World War II, Army psychologists attempted to assess the judgement of soldiers (Northrup, 1989). These judgement tests consisted of scenarios with a number of alternative scenarios. Solutions rested on the person’s ability to draw on his common sense, experience, and general knowledge, rather than logical reasoning.
In an influential study by Chan and Schmitt (2002), data from 160 civil service employees were analysed, with a view to demonstrating the validity of the SJT in predicting overall job performance as well as three performance dimensions: task performance (core technical proficiency; problem analysis, written communication, oral communication), motivational contextual performance (job dedication; motivation to perform, motivation to learn, motivation to work hard), and interpersonal contextual performance (interpersonal facilitation; conflict resolution, negotiation, teamwork and co-operation). Chan and Schmitt (2002) also felt that situational judgement tests provided incremental validity over prediction from cognitive ability, personality traits, and job experience.
Previously, Huffcuttt and colleagues (2001) had attempted to elucidate the most suitable construct categories. Their constructs included mental capability, knowledge and skills, basic personality traits, applied social skills, interests and preferences, organizational fit, and physical attributes.Recently, Christian, Edwards and Bradley (2010) argue that many studies have failed, however, to report the constructs actually measured in SJTs. A construct-level focus in the situational judgement test literature is therefore lacking. Christian and colleagues (2001) found that situational judgement tests most often assess leadership and interpersonal skills and those situational judgement tests measuring teamwork skills and leadership skills have relatively high validities for overall performance.
There has been an increasing drive to standardising these tasks. For example, the LR SJT has questions in written format. There is inevitably a difference for oral questions, or SJTs in video format (it is claimed that video SJTs have more subtle nuances involving social cognition or emotional intelligence which can be picked upon). Also, the questions can potentially vary in scenario length (longer scenarios tend to have more detail), and how scenarios are profession-specific (for example, type of firm, law/medicine/business). Finally, the questions can vary in format. In the LR SJT, and for example in the 2010 and 2011 Clifford Chance SJT, you have to pick one best out of the options given. In some tests, rather, you may be required to rank your choices in order of preference.
References
Chan, D, Schmitt, N. (2002) Situational judgment and job performance. Human performance, 15(3), 233-254.
Christian, MS, Edwards, BD, Bradley, JC. Situational judgment tests: constructs assessed and a meta-analysis of their criterion-based constructs. (2010) Personnel Psychology 63: 83-117.
Huffcutt, AI, Conway, JM, ROTH, PL, Stone, NJ.. (2001) Identification and meta analytic assessment of psychological constructs measured in employment interviews. Journal of Applied Psychology 80: 897-913.
McDaniel, MA, Hartman, NS, Whetzel, DL, Lee Grubb III, W. (2007) Situational judgment tests, response instructions, and validity: a meta-analysis. Personnel Psychology 60, 63-91.
McDaniel, MA, Morgeson, FP, Finnegan, EB, Campio, MA, Braverman, EP. (2001) Use of situational judgment tests to predict job performance: a clarification of the literature. Journal of Applied Pyschology 86(4), 730-740.
Weekley, JA, Polyhart, RE. (2006) Situational judgment tests: Theory, management, and application. Mahwah, NK: Erlbaum.
Two 'Legal Recruit' books on online tests for law students, pub date 1 Nov 2011
It is important to note that, whilst ‘Legal Recruit’ is an important key initiative from the BPP Legal Awareness Society to encourage a business culture in law, ‘Legal Recruit’ is absolutely nothing to do with BPP media, nor indeed represents any official teaching or guidance from BPP.
Book 1 – Practical verbal reasoning questions for law students (111 pages)
This book has carefully designed verbal reasoning questions, of the ‘True’, ‘False’, or ‘Cannot Say’ variety. Two questions follow each of the passages together with full explanations, and reading passages cover a variety of subjects, including biology, business, economics, education, engineering, environment, geography, geology, health and safety, human relations, medicine, modern languages, physics, technology, and transport. Readers of this title will benefit from the general explanation as to how to do these tests, and from the worked examples, such that they feel much more comfortable when they come to do such tests for real for training contract or vacation placement applications. The book will also be also of interest for applicants to corporates who use these tests for recruitment purposes. This title has a publication date of 1 November 2011, and is only available to download for £7.50 from the Legal Recruit website.
Book 2 – Practical situational judgements questions for law students (77 pages)
This book has carefully-designed questions in six competences commonly assessed in situational judgement tests. Situational judgement tests are used by some law firms to ascertain the suitability of a law student for a training contract. These competences are problem solving, proactive attitude, commitment to excellence, communication and negotiation, teamwork, and attention-to-detail and leadership. The book will also be also of interest for applicants to corporates who use these tests for recruitment purposes. Getting focused on these competences will help law students to understand the relevance of these skills to recruitment and their general professional life. This title has a publication date of 1 November 2011, and is only available to download for £7.50 from the Legal Recruit website.
A new free sample practice test to help law students excel at online applications assessments
I will be launching my new website ‘Legal Recruit’ to help law students become really good at two of the most important online assessments for legal recruitment.
These are currently verbal reasoning tests and situational judgement tests.
It’s essential you practice these before doing them ‘for real’ through online training contract and vacation scheme applications in the City corporate firms.
I believe that it’s a shame for students to come a cropper on these tests, and not get called for interview, even if they meet interview requirements elsewhere, such as II.1 or relevant vacation scheme placement. A colleague of mine – very well-known, in fact – refers to this phenomenon as “death by spreadsheet”.
The new website is here http://legal-recruit.org/.
I have been working extremely hard on it the last few weeks. Please feel to look around. Some of the features – like the main assessment ‘zone’ – are disabled, as I have not formally launched this yet.
For those who are interested in some of the relevant theory and evidence underlying this new website designed for law students, you might like to go to this blog http://legalrecruit.org/. I keep this blog up-to-date with useful info. There’s also a twitter thread on which I put up interesting stuff, including stories of the type which could easily feature in a verbal reasoning test, ranging from geology to technology: http://twitter.com/#!/legal_recruit.
If you go to http://legal-recruit.org/, you will see there’s a toolbar ‘about the tests’. If you go there, you’ll then see another toolbar, and if you go to any of the pages there, you’ll be able to download a free factsheet.
If you click register http://legal-recruit.org/register/ you will then be invited to sign-up. Sign-up involves you suggesting a username, and you’ll be sent a password by email to an e-mail address you suggest. You can then use the username and password to login (using the login button in the top-right corner) http://legal-recruit.org/wp-login.php.
Once you login, you’ll be able to do the trial http://legal-recruit.org/trial/. You should only do the trial if you are indeed of graduate level, i.e. you have graduated.
This is a practice test which gives you a very good idea how a real asssessment of the type routinely done for real training contract applications. After you do this test, you’ll be able to look at a full report as to how you do did.
You’ll know I have a bugbear about legal recruiters not entering into the letter of the law (or spirit more often) regarding ‘reasonable adjustments’ for tests. This website will allow you to alter the text-size meaning that some law students will not needlessly suffer in struggling to read the questions. For the practice assessments to be offered on my site from 1 November 2011, registered users will be able to vary the time that they can take for each test from 19 minutes to 39 minutes. You can read more about this here if you’re interested.
After taking the test, you’ll be confidentially be able to read the report on this link (if you’re logged in). Don’t worry – nobody else gets to see this report, not even us!
ALL OF THIS PRACTICE IS FOR FREE – ENJOY!
LegalAware four-part series on psychometric tests: Factsheet 3 – What are situational judgement tests?
This is the third in a four-part series looking at psychometric testing.
Lots of employers use situational judgement tests (SJTs) during their recruitment stage, particularly at graduate level.
SJTs measure your behaviour and attitudes to work-related scenarios. With a bit of “insider knowledge” you will have nothing to fear. This is where it is most likely to have a substantial advantage to ‘understand’ the corporate environment or culture, from having done ‘relevant’ work experience. Possibly reading all the books in the world about what it is like to work in a City environment won’t fully prepare you for such tests. However, here is one very good book, which is available on Amazon UK:
It is therefore perfectly possible for you to ‘train’ to become good at these tests. Currently organisations as diverse as Waitrose, the NHS, PricewaterhouseCoopers, Sony, Wal-Mart, Deloitte, John Lewis, the law firms Herbert Smith and Eversheds, the Fire Service and many more, are using SJTs as part of their recruitment process.
Situational judgement tests present candidates with a range of different situations that they might experience in the job for which they are applying. For each situation, a number of possible actions are suggested. There may be around 3 actions, but this varies. It is the candidate’s job to choose between these possible options and judge which is the most effective course of action to take and therefore which action they would take if faced with this situation.
SJTs are always multiple-choice; no answers other than the options listed are allowed.? The ‘name of the game’ is not to have an in-depth discussion over the various pros and cons of the various options on offer; often one of the options will be completely daft according to most reasonable people.
The situations (or “scenarios” as they are sometimes called) are almost always reflective of a real-life aspect of the job. SJTs are usually designed using ‘Subject Matter Experts’ – usually people who are successful at the job themselves. These experts are asked to suggest likely scenarios with which a jobholder might be faced and also to suggest possible responses and rate these responses for effectiveness. This forms the basis of the scoring system for the test. So in other words, how closely your responses match the answers rated highly by the ‘experts’ will determine how well you do on the test.
For employers, SJTs are a very cost effective, powerful and convenient way to select the potential strong performers from a large group of candidates. Employers will be more likely to use an SJT if they have a high volume of candidates applying for a role or position and if they recruit for this position on a regular basis.
How should you prepare for a SJT?
It has been suggested that one of the best ways to be prepared for a selection test, including a situational judgement test, is to be aware of what the test is seeking to measure. In other words, what aspects of you, as a candidate is the test hoping to pick up on?
Competencies are bundles of skills, abilities and personality traits which are considered by most experts to contribute to good job performance. The relevant competencies will vary according to the job or job-type being considered. As a law student applying for a training contract, you will normally be expected to demonstrate “graduate level competences”.
Graduate competencies will reflect the range of skills, abilities and styles that are effective at a graduate entry level role in an organisation.
They are unlikely to include managerial competencies such as ‘directing others’ and ‘strategic thinking’.
They will probably include some, or all, of the following:
- Communicating, influencing and negotiating – looking for clarity, appropriateness and persuasiveness? of communication.
- Drive to achieve results – looking for motivation and drive to achieve high standards and deliver results on time.
- Planning and organising – looking for the tendency to approach tasks in a systematic and organised fashion, to prioritise activities and manage time.
- Analysis and decision-making – looking for accurate and timely analysis of information, facts and data and good judgement with regard to what course of action to take based on that information.
- People and relationship skills – looking for capacity to build effective working relationships, to have empathy and awareness of others and work well in a team.
No particular training or knowledge is required to take this type of test. However, as mentioned above, if practice tests are available on the employing organisation’s website, or elsewhere, it is well worth taking full advantage of these.
When you sit down to take the test, look closely at the detail of both the situation, the possible answers, what you are being asked to comment on and also whether you are being asked for your judgement or information about your most likely response. It is important that you read each scenario thoroughly.
Another point is that, as for ability tests, you are expected to use only the information provided in the question; do not make assumptions about the situation or scenario, even if it is similar to one that you have come across yourself in the past.
And finally, as mentioned above, if you have been given information about the competencies assessed then keep this in the back of your mind as you progress through the test. If you haven’t been given this information then make your best guess as to the competencies that are typical of the role for which you are applying. By identifying the competency or competencies that the question is addressing you can more easily get into the correct ‘mindset’ to judge the options effectively.
Example questions
1. Your friend, who has always been your competitor at law school, is about to give a Powerpoint presentation on share acquisitions in Korea as a trainee, and you know that the Managing Associate is looking forward to this presentation with interest. However, there appears to be a mechanical fault with accessing Broadband, and the only copy of the file is an email which she sent to you to check yesterday. You saved it on your memory stick, which you happen to have brought to the meeting. You know the memory stick is compatible with the computer she is using for her presentation. How do you decide to proceed?
Pretend you have forgotten the memory stick, and you cannot help. (0%)
Offer to upload the presentation on her computer using the memory stick, but to offer also to download the file from the internet if that fails from a neighbouring computer. (94%)
Ask the Managing Associate for help, to demonstrate that you enjoy teamwork. (6%)
2. You are a trainee in the corporate finance seat in London where all team members are extremely busy. You have recently been liaising with ten particular clients on an almost daily basis in France. Your Supervisor has asked you to canvass for opinions of various clients in different countries towards the recent fall in stock prices in the European markets. You feel you do not have time to do this task on your own in time. Which of the options do you consider first?
Seek help from other trainees to help you to write the report, and ask other trainees which clients should be contacted. (24%)
Seek help from other trainees to help you to write the report, and contact some or all of the ten clients to ask them for their opinions. (47%)
Research the information which could be obtained from the clients and punctually write a report. (28%)
