Test Strategies: Computer Based and Computer Adaptive Exams
What are computer based tests?
How are CBT Different from Paper Exams
What are Computer Adaptive Tests
How Computer Adaptive Tests Work
Computer Adaptive Test Strategy
What are Computer Based Exams
CBT 101: Computer Based Exams are a tool to evaluate students that’s becoming more and more common, mainly because of its advantages over the traditional pencil-and-paper tests. They are also known as Electronic assessment, online assessment, computer assisted assessment or e-assessment. These tests are used for all sorts of assessments; psychiatrics, health, aptitudes or knowledge, meaning that they can be summative, formative or diagnostic. In education is mainly used to measure the cognitive abilities of a subject after a learning process.
A few examples of CBT and CAT are, TASC, PSB/HOAE
How are CBT different from Pencil and Paper Exams?
The main difference is that these tests are taken trough a digital platform, either online or as a download file. One of its greatest advantages is that they can be taken from anywhere and from any computer, so you can use your own from your home, or the ones available at the computer lab of your College.
These exams are in effect just like traditional paper-based exams; standardized tests with several questions in a specific order, related to a particular niche that the student must answer according to his/her knowledge. You can even choose to skip a particular question, in case that you don’t know the answer, and come back later to it, just like you can do with traditional test.
Another advantage of these tests is that they are computerized, so they are programmed to automatically add the score obtained by the student in each question, giving the final score almost immediately. Because of this computerized score the teachers and institutes save lots of time and assures that the score is error proof. As they are programmed tests they are much more secure and “cheating-proof”, making it another plus for the teachers. Also they are eco-friendly because it represents a considerable paper saving.
CBTs and other types of computerized assessments exist since the 70´s, but they weren’t too popular because the access to computers was limited, and not too many schools and colleges had a good computer lab. But now, when a computer is as part of a house as the kitchen, the circumstances have improved and the application of these tests doesn’t represent any mayor labor. So, it has been progressively expanding.
What are Computer Adaptive Tests?
CAT 101: On the other hand, there are Computer Adaptive Tests. These tests are also taken from any computer with internet connection or the necessary files. And, as any computerized test, they calculate the grade immediately, giving an error-free score. But what differentiates them from the CBE is that they aren’t standardized tests. This, of course, means that students won’t have to face the same questions in the same order.
The CAT works with a programming that allows it to “know” the level of difficulty of each question. So, if the student answers correctly, then the next question will be harder and the level of difficulty will be increasing. But if the student answers incorrectly then the next question will be easier.
There’s the source of the name and why they are also known as Tailored Testing. Because it fits the student´s profile, like a tailored suit to his body. This test adapts to the level of knowledge of the student by creating a profile based on his previous answers. According to these it determines the strengths of the taker and adjusts to his/her level of ability with great precision.
Their algorithm works like this; there is a “pool” of questions, each one with a defined level of difficulty. The test begins with an intermediate level of difficulty and, according to the answer given by the taker, it decides whether the next question must by harder or easier. With each answer the algorithm updates the examinee’s profile and keeps going from this point.
How Computer Adaptive Tests Work
The CAT has five components:
- A pool of calibrated questions. This must be available for the test’s software, so it can select any item it needs from it. It must be already calibrated, which means that every question must already have an assigned level of difficulty.
- A preliminary assess. By default the test’s software assumes that the taker has a medium level of abilities, so the first questions are not actually “adapted” to the student.
- Item selection algorithm. This is where the magic happens. This algorithm is the one that determines the abilities of the taker and selects the next question to be displayed. If the algorithm functions fine then the test will adapt to the taker.
- Automated Scoring procedure. According to the answers given by the takers the CAT calculates and estimated of the examinee’s abilities, giving a score that fits its criteria. The CAT uses two methods for it; the maximum likelihood estimations, which is a statistics model that estimates the results using the parameters of the model, and the Bayesian estimates, that uses the Bayesian theorem to calculate the accuracy of the results.
- Termination criterion. The algorithm is coded to repeatedly select a new item according to the previous answer infinitely or until every item of the pool has been used. For this not to happen the CAT has a termination criterion, that decides when the test is over. Usually the test ends when a precise score or level of abilities is determined.
These innovations in the assessment field are great, and have much more advantages than disadvantages. But a scenario where traditional paper-based exams are substituted by computerized tests is far away. There are technical difficulties that stand on the way, like having the necessary IT hardware to allow a large number of students to take an electronic examination at the same time, and the need to guarantee a rigorous level of security.
Computer Adaptive Test Strategy
Such a drastic change in how the test is given may be difficult for those who are used to take paper-based exams. Starting is harder, and students need a moment to get used to it. Here are a few tips and strategies for taking Computer Adaptive Tests:
Study! This might seem a bit obvious, but it is the primary and most important thing to do. If you know the topics for the test you’ll know the correct answers, and if you answer correctly you’ll have a good grade, it is as simple as that. It doesn’t matter if the test is computerized or paper-based, your best tool will always be your own knowledge. Take a look at your study habits, assess them and correct what needs to be adjusted. There are tons of practical guides in the web so you don’t have to suffer finding out what’s wrong with your methods. This way you’ll be armed for your exam. And you will not only be prepared to answer correctly the questions, but you will also feel confident and your nerves won’t be able to play tricks on you.
Familiarize! Computer-based tests are a new tool for students, it’s only natural that you don’t feel comfortable with something so unfamiliar, especially if you have used paper-based test your entire student life. The encounter with such a different platform might be bumpy and weird. That’s why you have to make sure that the first encounter won’t be the day of the test. Take your time to get to know the platform, practice with it, so you can feel comfy in it and not like a fish out of water. Most CBEs and CATs have practice resources available on the web for free, used them and incorporate them in your study routine. Practicing with the tests models will give you the chance to see the types of questions that are asked, measure the time it takes you to answer, see what aspects of your performance need to be enhanced and understand the platform before the big day comes.
Use the tutorials. Video tutorials are one of the greatest study tools that have emerged in the last few years. They are a great way to “attend” to a particular class and learn at your own pace. There are tutorials about virtually everything, including CBEs and CATs. Many of them have their own tutorials, designed to get you through the test step by step. If the exam you’re taking has one, make sure you go through it from top to bottom. Do it at your own rhythm and don’t rush it, pause when needed and go through it again if you have any doubts. Having you test explained step by step is a great support, and it will help you to understand what you need to improve and be ready for your exam.
Get there early! This is a basic that can give you several advantages. First of all, it is rude to be late at any date, it´s disrespectful and it says a lot about your character (not good things, by the way). Punctuality is a great presentation card. If you get to the place of the test early you’ll have time to calm your nerves, get familiar with the class room or lab where you´re taking the test and get to know your computer a little bit. Serenity is a key piece to clear thinking. Besides, to be on time means that you will be there when all the instructions are given. This is more important that you might thing, because every test has its particularities. Also you’ll have the opportunity to ask questions if you have them and start you exam with a clear head.
Learn the rules. Pay special attention to the instructions given at the beginning. Every test has its particularities, some allow you to skip around, while others don’t, some others have specific times for breaks, some others require you to log out for a break, and some others don’t allow you to log out until you are finished. These specifications are important for your performance and grades, so listen to the guidance, make sure you know the rules for your test and follow the instructions to the letter. Remember that is a computerized test, so if you submit information or answer wrongly you won’t have the chance to undo it. Don’t make dumb mistakes that can mischief your score for being uninformed.
If you have questions, ask. There is no reason to be shy. The professors, teachers and tutors are there to help you go through you’re test as best as you can. So ask any question you have. It doesn’t matter if you think that it might be a dumb question, or if you think is not that important. It´s not dumb and it is important. A doubt can harm your performance, or make you slower. Don’t let something with such a simple solution be a step back in your performance. If you have questions then ask them, period. And by being the brave one you might even help another taker that is too shy to ask the same question you have. Of course, it goes without saying that I mean questions regarding the digital platform or the test’s format, not about the actual subject that is being evaluated.
Use scratch paper. This is especially useful with math and science tests, but still can help you with any test. Have a blank paper where you can take notes about questions, do your calculations, write relevant details that you might use later, practice a diagram o graphic, take note of a specific question that you’ll review later, any other thing you need to remember or anything else you feel will help you answer the questions that follow. But still, you have to be careful, because this is not allowed in every computerized tests, it depends on several aspects. Another reason to know the rules of the test you’re taking. Most tests allow paper to take notes. Make sure you take a pencil, eraser and blank pages, in case they are not provided.
Concentrate on your answer, not in the algorithm. Some students concentrate on figuring out how the system works, to try going ahead and predicting the next questions. Some even research about the algorithm and the ways to trick it. Don’t worry about this, this is high level coding, there are plenty of people working to make the test secure and cheating-proof. It´s useless to invest effort trying to predict a system that is designed to be unpredictable. Instead use your energy and brain to study and answer correctly. It´s the only way to assure a great score.
Be aware/beware of the time. Another basic. Every test has time limitations, and the management of the given time is actually one of the abilities to be assessed. Most computerized tests have a timer that indicates you how much time left you have. Some are timed by question, others give you an overall time for the entire test. Typically, there is an on-screen clock that counts down the remaining time and alerts you when you are down to five minutes.
Anyway, you must take the time into consideration even when you’re practicing. Be aware of your abilities in every topic, so that you know which questions you’ll need more time. Don’t spend too much time on one question, if you don’t know the answer skip it and come back to it later (if it’s a CBE). In the case of CATs, which don’t allow you to skip and come back later, don’t stay on one question more than necessarily. So, answering fast is your best choice, even if the answer is wrong, because CATs are scored based on three components, how many questions you answered, how you performed on the questions answered, and statistical information for the questions answered. So, a slow answer can do more harm than a wrong one. Still, this doesn’t mean that you should answer anything just to be fast. Just take the necessary time to think your answer, but don’t waste time if you already know that you don’t know the answer.
Written by:
Modified: April 25th, 2018
Published: March 7th, 2007