Created by TestInvite / May, 2024
In Assessment Systems, questions and question types are used to assess the competencies of the candidates. Candidates are asked questions in a specific test, personality inventory or exam format, and in line with the answers given, first assessment and then evaluation are carried out according to a predetermined system.
Different competencies can be measured more successfully and efficiently with different question types. Therefore, in assessment processes, multiple and different different question types suitable for the process are conducted. Using the question type suitable for the competency to be measured and making use of different question types suitable for the process makes the assessment process of a competency more valid and reliable.
Question types in Assessment Systems are determined according to five different approaches.
The first approach used in determining the types of questions is to divide the questions into question types according to which method and how they are answered. So, rather than what we ask, how we want to get the answer (how the question is answered) determines the type of question.
A question can be answered in many different ways. Even if writing, speaking, choosing, ordering, matching are the most common answering methods, different answering methods are also used in computer-assisted assessment systems.
While the system provides the necessary assessment environment and tools for the candidate to create their own answer, on the other hand, it evaluates in what conditions and how successful the candidate will be, together with the information defined before the process, regarding the evaluation of the answer.
In this case, since the evaluation will be made by an objective criterion, impartiality is preserved. If an automatic evaluation criterion is not defined, the answers given are evaluated by an evaluator.
In multiple choice questions, a number of options are given to the candidate. The candidate answers the question by choosing or making choices among these options. It can be structured in many different ways, such as a true-false question, multiple choice question with only one correct option and multiple choice question with multiple correct choices.
In multiple-choice questions where more than one option can be selected, the options can be given weight, and gaining success can be ensured in proportion to the accuracy and weight of the selected options.
With multiple choice questions, the tendency of the candidate to different competencies can also be measured. For this, options are associated with trends.
In the matching, ranking questions the candidate is presented a number of items and options that can be matched to the items. By choosing one of the options given for each item, The candidate makes a matching or sorting and answers the question.
In this question type, in which the candidate matches each item with the options presented, each item can have its own unique options, or all items can have a common pool of options. In this case, the maximum number of items that an option can be matched with can be determined.
Some of the things you can do after the item indicated by the arrow is added to the question;
In open-ended questions, an environment is provided and tools are for the candidate to create the answer to the question. It can be made to respond in many different ways, such as writing text, writing numbers, creating content, filling in the blanks in a given table, coding, recording your voice via microphone or video via the webcam attached to the device, uploading files, taking photos, recording your screen. Each answering method can be thought of as a different type of open-ended question.
If the correctness or inaccuracy of the answers to open-ended questions can be determined automatically by a rule to be defined to the system, these questions are called Objective Open-Ended Question, and if not, Subjective Open-Ended Question. In subjective open-ended questions, the answers given must be evaluated by an evaluator.
For write-questions, the candidate is provided with a short or long text writing area where he or she can write his answer. The candidate answers the question by typing his answer in the required field using his own keyboard or using a virtual keyboard provided by the application.
Predetermined restrictions can be defined to the answers that can be written. In this way, the candidate can only be compelled to give an appropriate response to the defined constraints.
In number writing questions, the candidate is provided with a space to write the answer in numerals and numbers. The candidate answers the question by typing the number he or she thinks is correct using the numbers on his keyboard or using a virtual keyboard to be provided.
A math editor is provided for the candidate to use while answering, and the candidate answers the question by typing the mathematical expression using the editor.
The candidate can write a number consisting of numerals or a number with scientific notation.
In the images below, you can examine how the question looks on the candidate screen when the space that can be answered in both ways is activated.
In speaking questions, a voice recording application is opened to the candidate where he/she can record his/her voice over a microphone attached to his/her device. The candidate answers the question by recording his/her voice for a while using the record and stop buttons on this application.
The candidate can start answering the question by speaking or stop recording through the button indicated by the arrow.
It can be determined how many audio recording trials the candidate can make, and how long the total recording can be.
For Video Interview questions, a camera recording application is opened to the candidate where he/she can record his/her own video via a webcam attached to his/her device. The candidate answers the question by recording the video footage using the save, stop buttons in this application.
In file upload questions the candidate is asked to create a file in response to the question asked and is given a space to upload the file. After preparing the file, the candidate answers the question by uploading the file.
For coding questions, a coding editor opens, where the candidate can write code. It is predetermined in which programming language it can respond. The candidate answers the question by typing the necessary codes using the coding editor.
In table fill questions or tabular questions, the candidate is given a table in which its cubicles can update content values, and the candidate answers the question by filling the blanks in the table.
For photographing questions, the candidate is presented with an application that can save a photo or screenshot and upload it as an answer. The question is answered by taking a photo or saving a snapshot via the webcam on your device.
Another approach that can be applied in determining the types of questions in Assessment Systems is to divide the questions into types according to how the answers given to the questions will be evaluated. According to this criterion, we can divide the questions into 2 main groups:
In objective type questions, the answers given by the candidates to the questions are evaluated automatically by the assessment system and in complete impartiality by computer algorithms according to predetermined rules. This way, impartiality, equality and objectivity are ensured at the highest level.
Multiple choice questions, matching and sorting questions, are examples of objective type questions. In these types of questions, the system can evaluate the given answer in an impartial and automatic way, since the information about which options, matching and rankings are correct can be defined in advance to the system.
Questions such as filling in the blanks, answering by writing short text, answering by writing numbers which are among, open-ended questions, can also be structured objectively because clear rules can be defined for the answers that can be given. The assessment system can make an objective true/false evaluation by comparing the text written in the answer with the correct answer defined in the system.
The fact that it is not in need of human evaluation and therefore can be evaluated instantly and effortlessly by the assessment system provides both speed and convenience, cost advantage, and an impartial , evaluation by avoiding the subjectivity of human evaluations. Thanks to these features, the most frequently applied, most valid and reliable tests, exams and inventories in the world consist of objective type questions.
The disadvantage of objective type questions is that they may be insufficient in measuring some competencies. The use of objective type questions does not bring success, especially in measuring competencies that require creativity. Many qualities such as being a good copywriter, making beautiful pictures, being persuasive, being consistent, and being reliable cannot be measured using objective type questions or can be measured with very indirect methods, but their validity is low. These qualifications are subjective in nature, meaning that their evaluation cannot be bound by a rule and each evaluator may evaluate them differently.
The answers given by the candidate cannot be evaluated automatically by the assessment system. All answers are evaluated individually by one or more evaluators. Therefore, it may harbor serious problems and inequalities in terms of impartiality and consistency.
It is recommended that evaluators use predetermined standard evaluation criteria in order to avoid the negative effects of subjective type questions. On the other hand, negativities can be reduced to some extent by using methods in which a given answer is evaluated by more than one evaluator. However, in any case, the evaluation of subjective questions lags behind objective type questions in terms of impartiality and equality.
Since the answers given to the subjective type questions have to be evaluated individually by the evaluators, these processes are slower and their costs are high. Despite all these disadvantages, the use of subjective type questions is very important in the measurement of some competencies. For example, to measure English Speaking proficiency, a question may be asked that the candidate can answer by speaking. The answer given can be evaluated by the evaluator adhering to the predetermined evaluation criteria.
Questions answered by writing long texts, questions answered by speaking, questions answered by creating a file are highly subjective type questions. The measurement of the success of the answers will not be determined automatically by the assessment system, and one or more evaluators will have to score the answers. Although applications in which some artificial-intelligence algorithms try to evaluate such answers, or at least position the evaluator as a supporting element, no real success has been achieved yet.
On the other hand, not all open-ended questions have to be subjective. As we have exemplified before under the heading of objective type questions, questions answered by writing numbers, filling in the blanks, questions answered by writing short text can be structured in objective type question form, as they can be evaluated automatically with predetermined rules.
During the preparation of the question, auxiliary materials such as pictures, reading pieces, listening pieces, videos and files on which the question will be built can be used.
A reading text is given with the question and the question is structured through this reading piece. The candidate first reads the given content, then answers the question asked over the content.
Along with the question, an audio recording that can be listened to is given and the question is asked through this listening piece. The candidate first listens to the audio recording given, and then answers the question based on the piece he/she has listened to.
Along with the question, a video that can be watched is given and the question is structured through this video content. The candidate first watches the video recording given, then answers the question through the video he watched.
The number of times the candidate can watch the video and whether he/she is allowed to jump back and forth while watching can be determined in advance.
Along with the question, a file is presented to the candidate which they can download to his/her device. The candidate first downloads the file, analyses it, and answers the question.
Along with the question, a mathematical equation or formula created with the math editor is given and the candidate answers the question through the equation.
A question can be structured on its own, or it can be positioned within a group of questions.
Single questions consist of only 1 question and all the auxiliary content needed to answer the question is included in the question. As an example, you might consider a single multiple-choice question asked through a listening passage.
In grouped questions, many questions are grouped together with one or more content. As an example, you can consider a reading passage and 2 multiple-choice questions that follow the reading text.
In grouped questions, the questions may lose their meaning when separated from the content. That's why grouped questions are often included in a grouped exam, test, or inventory.
We can divide the questions into 2 main groups according to what and how a question measures.
In success questions, the answer to the question can be true or false. If the candidate gave the most correct answer to the question, he/she can get full points, if he/she gave an incomplete answer, partial points, and if he/she gave the wrong answer, he/she can get zero or negative points. In success questions, the candidate answers the question in order to get the maximum success score from the question.
The points to be obtained by answering the success questions affect the total success score obtained from the test or exam and the section success score obtained from the section in which the question is located.
Success questions can be associated with one or more dimensions. If a question is associated with a dimension, with the success gained from the question the dimension success score being associated with is also affected.
For example, if you associate a question with the "Rooted Expressions" dimension, the success to be obtained from the question will affect the test and section score, as well as the "Rooted Expressions Dimension Success Score".
As a result of associating the questions with the dimensions, it is possible to measure many dimensions from an exam.
The purpose of the tendency questions is to make a measurement of the candidate's tendency based on the answer given to the question. According to the answer given, a measurement is made of which orientation the candidate is closer to. Unlike Success questions, Tendency Questions do not have right and wrong answers.
Tendency questions are generally used in personality tests and competency evaluation inventories.
In the Testinvite assessment system, question types are handled from many different perspectives. From the contents to be included in the preparation of the question, to the method of answering; from positioning the questions in groups, to evaluating the answers; It is possible to configure questions in many areas, from calculating success scores to measuring sub-dimensions and tendencies.
Testinvite provides all necessary question types together to manage an accurate, valid and comprehensive assessment process. You can get detailed information about usage by watching our video tutorials, you can start using the system and try the usage by opening a free account, or you can request an online demo from our experts.
