AUTOMATIC GENERATION OF ENGLISH LANGUAGE TEST QUESTIONS USING MATHEMATICA

This paper describes a computer algebra-aided generation of two types of English language tests, which further develops our recent work in this domain. The computer algebra system Wolfram Mathematica significantly advances the process of English language testing and assessment. The automatic generation of questions allows us to create a large set of equivalent questions of a certain topic based on a small amount of input values. This reduces authoring time during test creation, avails application of equal criteria and a fair assessment, and decreases the influence of subjective factors. In our previous work, we proposed methods for automatic generation of English language test questions. These were aimed at evaluating the students’ knowledge of lexical and grammatical structures found in the text using test questions that involved matching words and their meaning, matching parts of the whole, and finding synonyms, antonyms, and generalizations or specializations of words. This paper provides new methods for the automatic generation of English language test questions. This includes generating questions for testing the students’ knowledge of adverbs and adjectives, as well as word formation, especially with negative forms of adjectives. UDC Classification: 004.42 DOI: http://dx.doi.org/10.12955/cbup.v4.794


Introduction
Evaluation tests are important instruments for assessing various English language skills and can be applied at any stage of training, including entry or intermediate level, or as a final assessment.A fundamental requirement in this process is the availability of a large number of questions to dissuade "cheating" or the memorization of answers.Many institutions require a certificate in English language proficiency from applicants wishing to study or seeking employment in the field of education and business (COE, 2001).In addition, many countries require adult immigrants to demonstrate basic knowledge of the host country's language (or at least a basic level of the English language) before being granted residence, work permits, or citizenship (COE, 2008).Due to the large number of students and multiple testing requirements, preparing a sufficient number of test questions and their assessment is difficult and time consuming.Hence, the reason to investigate the possibilities for advancing the testing and assessment processes, as described below.
A convenient facility for test authors is the DisPeL (Distributed Platform for e-Learning; Rahnev, 2014aRahnev, , 2014b)), a specialized editor for defining parameterized questions, which are automatically assessed.DisPeL allows parameterization of test questions and examination problems in differing areas, such as accounting (Rahnev & Rahneva, 2008), SQL (Rahneva, Golev & Pavlov, 2008), electronic circuits (Rahneva, Rahnev, Pavlov, & Valchanov, 2005), evaluation of the financial efficiency of investment projects (Rahnev, 2014c), and the English language (Malinova & Ivanova, 2011).Despite the differing approaches required by the specifics of each subject area, the approach applied in all cases comes down to the redefinition of a single-answer/multiple-choice question as a dynamic parameterized one.The body of the parameterized question contains dynamic variables, which have values that vary in a user-defined range.Dynamic answers are described by a type, a formula for calculation and a comparison method that provides the correct answer.Students receive unique test questions by automatic generation of the values for the dynamic variables in the parameterized question.
Another tool we have used to generate test questions automatically is the Wolfram Mathematica system (Wolfram Mathematica, 2016).The following sections discuss the computer algebra aided generation of English language test questions and provide two examples of newly implemented algorithms.

Computer algebra aided generation of English language test questions
The aim of our recent work is to provide convenient tools for testing and research in the area of automatic generation of English language test questions.The computer algebra system, Wolfram Mathematica, is used as a fast and rich programming environment for research work and algorithm creation in this domain.In our first study (Malinova & Ivanova, 2011), we proposed a method for the parameterization of English language test questions of types dichotomy and multiple-choice with one or more correct answers.These are test questions connected with evaluating a students' knowledge of the lexical and grammatical structures that are found in the text.The algorithm implementations were added to the DisPeL editor for defining parameterized questions that are automatically assessed.The results demonstrated that, based on a small amount of input values for dynamic variables and patterns for parameterized sentences, we can generate a large set of equivalent dynamic questions of a certain topic/subtopic, chosen by the author.
In (Malinova, Ivanova, and Rahnev, 2015) we proposed a reimplementation of these results in Wolfram Mathematica in order to use this platform as a fast and convenient tool for the testing and implementation of new algorithms.In addition, we proposed methods for parameterization and automatic generation of English language test questions of the following types:  Test questions for matching words and their meaning;  Test questions for matching parts of the whole;  Test questions for finding synonyms, antonyms, and generalizations/specializations of words.
The following sections provide the new methods we have developed for the automatic generation of English language test questions using Mathematica.This includes generating questions for testing a student's knowledge of adverbs and adjectives, as well as that of word formation, especially negative forms of adjectives.
The presented examples use Mathematica's functions WordData and DictionaryLookup which fetch data form Wolfram's servers.Internet connectivity is therefore required for the code to work.

Automatic generation of English language test questions for testing adverbs and adjectives
Test questions about distinguishing adverbs and adjectives show students' knowledge of the definitions, function, and usage of these major parts of speech in the English grammar.With this question type, the students must select either an adverb or an adjective to complete the sentence.Mathematica's function WordData provides a number of special properties for adjectives: "BaseNoun", "DerivedAdverbs", etc.In this section we use WordData, invoked with the argument "DerivedAdverbs", which returns the adverbs derived from an adjective.For instance: a={"easy","quick"}; WordData[#,"DerivedAdverbs","List"]&/@a returns:

{{easily},{quickly}}
We have developed Mathematica code for the automatic generation of a large number of unique test questions for testing the diferences between adjectives and adverbs, based on a small amount of input data -8 parameterized sentences and 4 adjectives.Figure 1 shows the results in Mathematica.The correct answer is marked with parentheses.

Automatic generation of English language test questions for word formationnegative forms of adjectives
An important type of test questions is connected with word formation.This section provides examples of English language test questions for word formation of the negative forms of adjectives.To generate a large number of test questions automatically for testing knowledge of the correct negative form of adjectives, we used the function "DictionaryLookup" and a number of functions for string manipulation and combinatorics in Mathematica.DictionaryLookup finds all English dictionary words that match a given string pattern.The implemented code in Mathematica generates test questions as shown in Figure 2. Correct and incorrect answers are given below each question.

Conclusion
The computer algebra system Wolfram Mathematica significantly advances the process of English language testing and assessment.The automatic generation of questions allows us to create a large set of equivalent questions of a certain topic, based on a small amount of input values.This reduces authoring time during tests creation, avails application of equal criteria and fair assessment, and decreases the influence of subjective factors.The number of generated test questions can be significantly raised by adding single elements to the list of input values.
The new results, developed in Mathematica, can further be integrated back in DisPeL (or other etesting systems) and thus used for the automatic generation of evaluation tests and automatic assessment in English language education.
The results presented in this paper are used in English language education at the Faculty of Mathematics and Informatics of Plovdiv University "Paisii Hilendarski", Bulgaria.

Figure 1 :
Figure 1: The generated test questions for adverbs vs. adjectives