Data mining technology for the evaluation of learning content interaction.

Data mining technology for the evaluation of learning content interaction. Interactivity is central for the success of learning. In e-learningand other educational multimedia environments, the evaluation ofinteraction and behaviour is particularly crucial. Data mining, anon-intrusive, objective analysis technology, shall be proposed as thecentral evaluation technology for the analysis of the usage ofcomputer-based educational environments and in particular of theinteraction with educational content. Basic mining techniques arereviewed and their application in a web-based third-level courseenvironment is illustrated. Analytic an��a��lyt��icor an��a��lyt��i��caladj.1. Of or relating to analysis or analytics.2. Expert in or using analysis, especially one who thinks in a logical manner.3. Psychoanalytic. models capturing interactionaspects from the application domain (learning) and the softwareinfrastructure (interactive multimedia) are required for the meaningfulinterpretation of mining results. ********** MOTIVATION Interactivity is a central concept in educational environments(Sims, 1997). It refers to the interaction of a learner with thelearning material, the instructor, or with peers in the process oflearning (Moore, 1992). Interaction in different forms is known to bebeneficial for the learning experience and the overall effectiveness oflearning. However, a clear definition is still not unanimously agreedupon Adj. 1. agreed upon - constituted or contracted by stipulation or agreement; "stipulatory obligations"stipulatorynoncontroversial, uncontroversial - not likely to arouse controversy , which makes the instructional design Instructional design is the practice of arranging media (communication technology) and content to help learners and teachers transfer knowledge most effectively. The process consists broadly of determining the current state of learner understanding, defining the end goal of for and the evaluation ofinteractive teaching and learning environments issues of ongoingconcern. A fundamental question in relation to instructional design in thecontext of e-learning and other computer-supported teaching and learningis how learners interact with educational multimedia; that is, whattheir concrete behaviour, what their preferred learning style, and whattheir learning goal in such an educational environment is. Answers tothese questions are important for integrated formative evaluation Formative evaluation is a type of evaluation which has the purpose of improving programmes. It goes under other names such as developmental evaluation and implementation evaluation. andinstructional design (Hirami, 2002). The questions are, however, moredifficult to answer, if, as in the case of computer-based teaching andlearning, direct contact between learner and instructor or betweenlearners is reduced and less feedback is available for the instructor(Northrup, 2001). Another difficulty is created by the potential of theWeb and other educational multimedia to enable novel and innovativeforms of teaching and learning that are, consequently, not always wellunderstood (Ohl, 2001). Data mining (Chang, Healy, McHugh, & Wang, 2001) shall beproposed as the central, observation-oriented evaluation technique,which can provide an important contribution to the understanding oflearner interaction and instructional design. Data mining is used in avariety of domains from business-oriented decision support systems toscientific data analysis. Data mining is a technique that allows thediscovery and extraction of latent Hidden; concealed; that which does not appear upon the face of an item.For example, a latent defect in the title to a parcel of real property is one that is not discoverable by an inspection of the title made with ordinary care. knowledge, such as learners'behavioural Adj. 1. behavioural - of or relating to behavior; "behavioral sciences"behavioral patterns and usage rules in interactive educational systems,from a computer system's access logs. With data mining, essentialactivities can be captured, learner behaviour determined, and thisbehaviour interpreted in the context of learning styles and goals. Theadvantage of data mining over classical evaluation techniques such assurveys and observations is that it is an objective, non-intrusivetechnique that allows constant monitoring and evaluation at any time. The ubiquity UbiquitySee also Omnipresence.Burma-Shavetheir signs seen as “verses of the wayside throughout America.” [Am. Commerce and Folklore: Misc. of Web and Internet technologies, which are now thepredominant pre��dom��i��nant?adj.1. Having greatest ascendancy, importance, influence, authority, or force. See Synonyms at dominant.2. technologies for computer-based teaching and learning(Weston & Barker barkera term for an animal that does not usually bark which makes a violent respiratory effort, often during a convulsion, accompanied by a sound which roughly resembles a dog's bark. , 2001), has resulted in a special form of datamining called web mining, the use of data mining in Web and Internetsystem analysis and evaluation. Web mining has been used extensively ine-commerce applications to analyse an��a��lyse?v. Chiefly BritishVariant of analyze.analyseor US -lyzeVerb[-lysing, -lysed] or -lyzing, customer behaviour. It has proved apowerful tool in improving web site structures, managing customerrelationships, and in providing personalised Adj. 1. personalised - made for or directed or adjusted to a particular individual; "personalized luggage"; "personalized advice"individualised, individualized, personalized features: achieving bettershopping experience for customers and improved sales for companies. Webmining, however, has, with a few exceptions (Zaiane, 2002; Pahl &Donnellan, 2002), not been deployed in e-learning, despite the potentialof being equally valuable in improving the learning experience forlearners and in enabling an improved learning process for instructors.Content interaction, central in Web-based teaching and learning, shallbe our focus. The key objective here is to introduce data mining technology anddemonstrate its potential for e-learning and other computer-supportedteaching and learning systems. E-learning differs from, for instance,e-commerce in that learning as the goal is more complex and, based oncognitive processes Cognitive processesThought processes (i.e., reasoning, perception, judgment, memory).Mentioned in: Psychosocial Disorders , more difficult to capture than shopping and sales.Interactivity is the central concept. Therefore, educational models ofinteractivity and learning, but also the technical aspects ofinteraction of learners with educational multimedia software arecentral. How these different approaches to interactivity can beintegrated and applied as a data mining-supported analytic model for theevaluation of learning shall be illustrated. In order to benefit fromthe full potential of data and web mining, we need to be clear aboutwhat exactly the evaluation goal is and what types of learning-relatedinformation we can obtain for an analysis. Principles of learning andinteractivity, in particular in computer-based educational environments,need to be understood and made explicit. Investigating interactivity inall its aspects is a necessary prerequisite pre��req��ui��site?adj.Required or necessary as a prior condition: Competence is prerequisite to promotion.n. for the successfuldeployment of interactivity mining techniques. PRINCIPLES OF DATA AND WEB MINING Data mining is concerned with the discovery and extraction oflatent knowledge from a database (Chang, Healy, McHugh, & Wang,2001). Typically, this knowledge is classified into rules and patternsthat can help an analyst in analysis and decision making processes. Datamining has been used for a wide range of applications ranging fromdecision support systems in business applications to analysis tools inscientific applications. The purpose of data mining can be predictory(decision support), generative gen��er��a��tiveadj.1. Having the ability to originate, produce, or procreate.2. Of or relating to the production of offspring.generativepertaining to reproduction. (create new/improved designs), orexplanatory ex��plan��a��to��ry?adj.Serving or intended to explain: an explanatory paragraph.ex��plan (scientific analysis). Web (usage) mining is the analysis of (user behaviour) data inweb-based systems. The database is the access log created by a webserver. The fact that only activities are recorded makes web usagemining See Web mining. different from data mining in general. Each web request of anytext document or other type of resource is recorded in the access log.Web log entries reflect activities: the requestor, access type, accesstime, and requested resource are recorded. In education-specific terms,the learner, form of activity/interaction, access time, and content itemare recorded. A range of classical data mining techniques exist that support theextraction of rules and patterns from a database (Agrawal & Srikant,1995; Cooley, Tan TANSee tax anticipation note (TAN). , & Srivastava, 1999; Chang, Healy, McHugh, &Wang, 2001): * Usage statistics are usually not considered as data miningtechniques. However, they often form the starting point Noun 1. starting point - earliest limiting pointterminus a quocommencement, get-go, offset, outset, showtime, starting time, beginning, start, kickoff, first - the time at which something is supposed to begin; "they got an early start"; "she knew from the of evaluations.For web-based systems, usage is captured in simple statistical measuressuch as total number of visits, number of visits per page, and so forth.Tracking features of most e-learning platforms are based on thesemeasures. * Classification and prediction are related techniques.Classification predicts class labels, whereas prediction predictscontinuous-valued functions. A model is used to analyse a sample. Theresult of this learning step is then applied. Regression is a typicalform of prediction. * Clustering groups mutually similar data items. In contrast toclassification, the class labels are not pre-given. The learning processis called unsupervised in this technique. Pattern recognition is atypical example. * Association rules are interesting relationships discovered amongthe set of data items. A typical example is purchasing analysis, whichcan identify item pairs frequently purchased together. * Sequential pattern analysis is applied if events are captured ina database over a period of time. Frequently occurring patterns areextracted. web usage or sales transaction patterns are typical examples. * Time series, the analysis of the variance of patterns and rulesover time, are important since they allow the analyst to evaluatechanging and varying behaviour. Often a session, which is a period of uninterrupted usage, is thebasic unit of analysis. The understanding of levels of abstraction In object technology, determining the essential characteristics of an object. Abstraction is one of the basic principles of object-oriented design, which allows for creating user-defined data types, known as objects. See object-oriented programming and encapsulation. 1. of knowledge and oflanguages to express knowledge is critical for the success of the datamining technology. Concepts, e.g., learning activities and interactions,have to be clarified. A web log contains low level concept information.Moving up the concept hierarchy of learning interaction means to movefrom a technical reflection of interaction to learning activities, tasksand goals. Summarising and generalizing individual behaviour intobehavioural patterns and the interpretation these patterns in adomain-specific analytic model is essential for an evaluation. In the implementation of the mining techniques, different phasescan be distinguished, (Figure 1). * After a cleaning phase, in which irrelevant entries such asimages in text pages are removed, rules and pattern extraction is thefirst core phase based on mining techniques introduced earlier on.Abstraction and retrieval functions are performed. Often the firstselection of patterns and rules has to be reduced to a tractable tractableeasy to manage; tolerable. size bythreshold control, identifying meaningful and interesting patterns andrules only. * Interpretation is the second core phase where patterns and ruleshave to be interpreted in an analytic model. Only in the context of theapplication domain can it be decided what a meaningful mining result isand how these results have to be interpreted. The interpretation istherefore guided by an explicit or implicit analysis model. If the focusis on usage and user behaviour as in web-based systems, languages areimportant in these analytic models. For instance, web log andinteractivity languages allow us to formulate formulate/for��mu��late/ (for��mu-lat)1. to state in the form of a formula.2. to prepare in accordance with a prescribed or specified method. rules and patterns of userinteraction. A few limitations apply to web mining. A classical problem iscaching. Web browsers The following is a list of web browsers. HistoricalHistorically important browsersIn order of release: WorldWideWeb, February 26, 1991 Erwise, April 1992 ViolaWWW, May 1992, see Erwise keep copies of already visited pages in a cache.Further requests of these pages will be satisfied from the cache and donot produce entries in the web server's access log. However,techniques to deduce de��duce?tr.v. de��duced, de��duc��ing, de��duc��es1. To reach (a conclusion) by reasoning.2. To infer from a general principle; reason deductively: the correct patterns exist (Zaiane, 2001).Furthermore, most advanced systems use dynamically created pages, whichcircumvents this problem. Some recent technologies such as mobile agentsrequire changes to the way access logs are created since computation Computation is a general term for any type of information processing that can be represented mathematically. This includes phenomena ranging from simple calculations to human thinking. ismoved from the server to the client. Web mining is limited to activitiesoccurring under control by the web server. Activities beyond that siteare not captured and cannot be mined. INTERACTION Interaction is here the central concept, a concept that has meaningon different layers ranging from the learning domain to the technicalinfrastructure. Three perspectives on interaction shall be offered: (i)interaction and learning, (ii) human-computer interaction Human-computer interactionAn interdisciplinary field focused on the interactions between human users and computer systems, including the user interface and the underlying processes which produce the interactions. , and (iii)interactive educational multimedia. Learning and Interaction It has long been argued that learning should be an active process(Dewey, 1916). Interactivity is an essential ingredient in this processthat positively affects learner satisfaction and performance. Eventhough various authors have addressed this issue, an accepted definitionof interaction for the various educational environments, in particularaddressing educational software, is still an open question. A model ofthe interactions that take place between a learner and what the learneris trying to learn can help instructional designers provide the learnerwith a diverse set of interactions that foster learning. Such a modelalso supports the instructor in the evaluation of interactive teachingand learning. Models provide the foundations for an approach toinstructional design; they clarify what the learning activities and whatthe knowledge representations are. [FIGURE 1 OMITTED] Some authors organise interaction by roles (Moore, 1992; Norman,1998). Moore introduces three types: learner-content,learner-instructor, and learner-learner interactions. Some attempts havebeen made to revise the role-based models in favour of a more content,or knowledge-based definition (Jonassen, 1994; Sims, 1997). Theinteraction with content, which represents subject knowledge, has acentral function, in particular if we move away from the classicalclassroom-based educational environment with teacher and peerinteraction towards computer-based education (Ohl, 2001). Interaction isan internal dialogue of reflective Refers to light hitting an opaque surface such as a printed page or mirror and bouncing back. See reflective media and reflective LCD. thought that occurs between learnerand the material. Interaction is triggered and supported by events inthe learning environment, such as interaction with computer-basededucational media. Ohl (2001) focuses on how the learner interacts withwhat is to be learned; these are often the only two core elements thatremain if we leave the classical educational environment. Human-Computer Interaction The definition of learning as a dialogue between learner andcontent needs to be adapted to the human-computer environment.Norman's (1988) model of interaction between human and computerintroduces an execution-evaluation cycle. The user formulates a plan,which is then executed by the system. The user observes the interface toevaluate the results and to determine further actions. We use models tocapture principles of interaction. A model's purpose can be bothevaluative (support in evaluation) and generative (support in design).Cognitive models The term cognitive model can have basically two meanings. In cognitive psychology, a model is a simplified representation of reality. The essential quality of such a model is to help deciding the appropriate actions, i.e. represent the knowledge, intentions, and processingabilities of users (Dix, Finlay, Abowd, & Beale, 1993). Inparticular, the acquisition and formulation formulation/for��mu��la��tion/ (for?mu-la��shun) the act or product of formulating.American Law Institute Formulation of plans of activity througha hierarchical model In a hierarchical data model, data are organized into a tree-like structure. The structure allows repeating information using parent/child relationships: each parent can have many children but each child only has one parent. of goals and tasks and the execution of plans ofactivity through a linguistic model need to be addressed. A hierarchical model captures the user's task and goalstructure. A task is an operation to manipulate concepts of the domain.A goal is the desired output from a task to be accomplished in somedomain. A goal and task hierarchy is defined by dividing goals intosubgoals and tasks. A user accomplishes goals by solving subgoals.Strategies define how goals on the same level are connected andscheduled. Strategies are the expression of learning styles and studyhabits. The acquisition and formulation of goals and tasks is based onknowledge-level attributes relevant to the user. This will then bemapped into actions and attributes relevant to the system. Data miningwill turn out to be ideally suited for the discovery and extraction ofgoals, task execution, and interaction behaviour. A linguistic model focuses on the user-system grammar constraining con��strain?tr.v. con��strained, con��strain��ing, con��strains1. To compel by physical, moral, or circumstantial force; oblige: felt constrained to object.See Synonyms at force.2. the interaction. This takes into account the different styles ofinteraction, direct manipulation, command line, or form fill thatcharacterise Verb 1. characterise - be characteristic of; "What characterizes a Venetian painting?"characterizedifferentiate, distinguish, mark - be a distinctive feature, attribute, or trait; sometimes in a very positive sense; "His modesty distinguishes him from his the form of manipulation of concepts represented byinterface elements. Languages that capture the process of interactionserve different purposes: for example to analyse the cognitivedifficulty of the language or to describe dialogues. A dialoguespecification, which captures the recurring re��cur?intr.v. re��curred, re��cur��ring, re��curs1. To happen, come up, or show up again or repeatedly.2. To return to one's attention or memory.3. To return in thought or discourse. cycles of execution andevaluation, defines what the legal user actions and system responsesare. The language is central since user activities and dialogues can beextracted from web logs through data mining. Cognitive architectures (architecture) cognitive architecture - A computer architecure involving non-deterministic, multiple inference processes, as found in neural networks. Cognitive architectures model the human brain and contrast with single processor computers. help us to address cognitive learningprocesses in the context of human-computer interaction. The architecturethat a system provides to allow a user to accomplish a goal defines aproblem space, the actions that can be used to traverse traverse - traversal this space, anda set of desirable states that represent the goal. The problem is thesubject; the goal is to learn about the subject. Educational systemsincorporate paths to successful learning; they characterise possiblesolutions to the learning problem in form of structure and dialogues.The user provides a solution how to traverse to the desired state. Wecan distinguish two levels of interaction and activity: knowledge-levelactivities, which are tasks and actions based on the user'sperception of the system, and problem-space processes, which involvegoal formulation, action selection, action application, and goalcompletion. Knowledge-level goals can be inferred from the formulationof problem space-level goals, for example through web mining. Behaviourat the knowledge-level can be inferred from the operation selection. Interactive Educational Multimedia In computer-based education, the interaction of a learner with whatis to be learned, presented by the educational software, is central.Usually, educational software provides the learner with various forms ofcommunication, often using different individual media. Therefore, we usethe term interactive educational multimedia to emphasise the interactivenature of educational software and the variety of activities andinteractions. Understanding educational software as interactivemultimedia systems is a key to understanding learning and interactivityin computer-based educational environments. Multimedia software isdesigned for interaction with the user and with usability How easy something is to use. Both software and Web sites can be tested for usability. Considering how difficult applications are to use and Web sites are to navigate, one would wish that more designers took this seriously. See user interface and usability lab. in mind(Elsom-Cook, 2001). Principles of multimedia systems explain issuesrelated to knowledge that is represented, activities that are offered,and types of interactions that are possible. Multimedia systems are characterised by the communication channelsthat are provided for users to access and communicate knowledge. Theuser uses specific languages to communicate along these channels. Theinterface that allows a (human) user to access and to communicate withthe system plays a crucial role. Channels and languages are centralelements in the communication between agents in a multimedia system. Anagent has an internal state, some goals and intentions, and the abilityto communicate. A key attribute of communication is modality modality/mo��dal��i��ty/ (mo-dal��i-te)1. a method of application of, or the employment of, any therapeutic agent, especially a physical agent.2. , thesensory system Noun 1. sensory system - a particular sensesense modality, modalitysensory faculty, sentiency, sentience, sense, sensation - the faculty through which the external world is apprehended; "in the dark he had to depend on touch and on his senses of smell and (auditory auditory/au��di��to��ry/ (aw��di-tor?e)1. aural or otic; pertaining to the ear.2. pertaining to hearing.au��di��to��ryadj. , visual, tactile tactile/tac��tile/ (tak��til) pertaining to touch. tac��tileadj.1. Perceptible to the sense of touch; tangible.2. Used for feeling.3. , etc.) through which thecommunication occurs. A channel is an abstraction of a connection devicethat makes the communication between two agents happen. Communicationneeds to be meaningful. We call a communication an interaction if itresults in a change of state of the other agent. A common language thatcan be written or read by the agents is a prerequisite for interaction.A medium is a set of co-ordinated channels, possibly spanning severalmodalities ModalitiesThe factors and circumstances that cause a patient's symptoms to improve or worsen, including weather, time of day, effects of food, and similar factors. , which possesses a cross-channel language of interpretation.The user interacts with the system in form of dialogues to accesscontent. During the usage of the system, the user creates a usuallyimplicit model that describes her/his preferred way of interaction andthat reflects goals and strategies. EVALUATION OF LEARNING AND INTERACTION BEHAVIOUR Behaviour and Learning Interaction Notions of interaction vary in different contexts: the learningprocess, interaction between human and computer, and communication inmultimedia systems. Interaction in the learning context refers toactivities of the learning process. Human-computer interaction separatesthe human knowledge-level and system-level aspects of interaction.Interaction in multimedia systems is based on technical notionsaddressing the communication of information and commands, which isreflected in a system's access log. These different notions of interaction, however, can be integratedfor educational software environments using models and languages.Activities that are central in learning style models are mapped to theservices of the educational multimedia system. An evaluation of theinteraction behaviour in the multimedia system can help us to determinethe user's preferred learning style. In general, learningactivities and educational multimedia services can be related. Patternsof learning behaviour, such as expressing study habits and strategies,can be associated with communication patterns. Evaluation Objectives The evaluation of the learning behaviour in interactive educationalmultimedia environments is the objective here. Data mining technology isthe key tool in this endeavour. It allows us to extract the interactionbehaviour of learners in the form of patterns and rules from accesslogs. These rules and patterns of interaction behaviour shall beinterpreted in the context of interactive learning. The formativeevaluation objectives are explanatory (to understand learning processesand to validate To prove something to be sound or logical. Also to certify conformance to a standard. Contrast with "verify," which means to prove something to be correct.For example, data entry validity checking determines whether the data make sense (numbers fall within a range, numeric data designs) and generative (to create improved designs andutilise knowledge in e.g. adaptive and personalised systems). Anassessment of the usability of the educational media is also possible. Similar to web mining for e-Commerce systems, where the analysisaims to support prediction and decision making for marketing and sales,the interpretation of mining results has to take place within ananalytic model. Here, the model has to capture the context of learningand interactivity. Interactive educational multimedia as the educationalenvironment need to be understood in order to interpret patterns andrules, and to integrate and utilise the results in evaluation andinstructional design. Concrete evaluation goals include: to identifynovice or weak learners, to monitor individual learners or learnergroups, to support usability analyses, or to carry out learner profiling(i.e., identifying learner characteristics) in order to determine auser's preferred learning style or to adapt a system to particularlearner groups. Languages and Models The behaviour is a manifestation man��i��fes��ta��tionn.An indication of the existence, reality, or presence of something, especially an illness.manifestation(man´ifestā´sh of the user's goals andstrategies (Mullier, Hobbs, & Moore, 2002). The expression ofinteraction and learning behaviour through a language is the key tool inthe evaluation of learner behaviour. Multimedia interface languages arebased on actions such as mouse clicks or keyboard input. The user'sinteraction with a web browser The program that serves as your front end to the Web on the Internet. In order to view a site, you type its address (URL) into the browser's Location field; for example, www.computerlanguage.com, and the home page of that site is downloaded to you. interface can be described in terms ofspecific interactions such as navigation and submission. Web logs are anexpression of behaviour, reflecting the interaction of users with aweb-based multimedia system. Interactivity models propose to distinguish different forms oflearning interaction. Ohl (2001) introduces knowledge acquisition (aninformation request that is answered by a system output, such asnavigation), knowledge production (information generation mostly in formof input to the system, such as submission), and operational acquisition(a combination of the first two). We capture the information flow andprocessing by an interaction topology topology,branch of mathematics, formerly known as analysis situs, that studies patterns of geometric figures involving position and relative position without regard to size. consisting of nodes and arcs.Nodes represent states of the systems; arcs represent transitionsbetween these states. State-transition notations are standard todescribe the dynamics of computer-based systems Computer-based systemsComplex systems in which computers play a major role. While complex physical systems and sophisticated software systems can help people to lead healthier and more enjoyable lives, reliance on these systems can also result in loss of . Astate-transition-based learning activity language describes alearner's interaction with educational multimedia: * Nodes represent the system's response including staticoutput (text, image), dynamic output (animation), and input facilitatingactivity (form, editor). Nodes are described by attributes includingmodality, subject, and educational service. * Arcs represent the activities offered to the user including links(navigation) and submission (action). Arcs are described by attributesincluding activity type (link, submit), action (the educationalactivity), action object (educational object), and source and target(nodes). For streamed media (audio, animations, and video) interactionstypically happen in form of actions such as start, stop, pause, forward,or rewind re��wind?tr.v. re��wound , re��wind��ing, re��winds1. To wind again or anew.2. To reverse the winding of (recording tape or camera film).n.1. The act or process of rewinding. . The goal, or the evaluation of learning activity, is realisedthrough the evaluation of access log expressions using miningtechniques. A language describing navigation behaviour as represented inaccess logs and a language describing learning activities andinteractions differ. In web logs, activities are not explicitlyrecorded, only the resource requested by the action. However, learningactivity expressions can be inferred from access log expressions. We canannotate annotate - annotation nodes of access logs with activity-related arc types. Thismetadata (1) (meta-data) Data that describes other data. The term may refer to detailed compilations such as data dictionaries and repositories that provide a substantial amount of information about each data element. language containing activity annotations is closer to thelearning activity language, web mining results based on annotated logscan be interpreted in a learning-specific analytic model. Sequentialpatterns identify learning tasks and activities, session classificationdetermines predominant learning goals, and time series identify learningstrategy and behaviour changes. [FIGURE 2 OMITTED] The learning activity language and its underlying concepts form theanalytic model in which the access log is interpreted. The analyticmodel reflects the designer's or evaluator's understanding ofinteractivity, learning activities and educational multimedia. Thislanguage is often not explicitly formulated for��mu��late?tr.v. for��mu��lat��ed, for��mu��lat��ing, for��mu��lates1. a. To state as or reduce to a formula.b. To express in systematic terms or concepts.c. . However, explicit languageand model would form an integrating component between evaluation andinstructional design for educational multimedia. WEB USAGE MINING FOR EDUCATIONAL MULTIMEDIA Web usage mining extracts rules and patterns from access logs, seeFigure 1. Different mining techniques support a variety of analysis andevaluation objectives. Session Statistics A session is a sequence of web log entries that reflects theinteraction behaviour of a learner in a period of active study. Somebasic measures can help to answer questions about the investment of timefor a given learning activity. Any of the results can be comparedagainst the expectations of the instructional designer. Explicitlyformulated expectations form part of the analytic model. There are otherstatistical measures that might result in useful insights. The totalnumber of requests by interval or total numbers ranked by resourceprovide relevant information. These measures, however, give more of anidea about "what" resources are used than "how" theyare used. Session Classification and Goal Identification The objective of this technique is to extract the main goal(s) andhigher-level tasks of the goal/task hierarchy from a session log(Donnellan, 2002). Typically, a learner focuses on one or two mainactivities in a session. Using a classification approach we can identifythe main learning objective(s) by looking at the predominant types ofinteraction with the system, Figure 2(a). The media resources of a course Web site can be classified, i.e.classes [C.sub.1], ...,[C.sub.N] are created where each class [C.sub.i]is a set {[U.sub.i1], ...,[U.sub.iM]} of URLs. Each class [C.sub.i] isassociated to a type of system-level interaction that facilitates aparticular knowledge-level learning activity, such as attending avirtual lecture or working on virtual lab exercises. If a learner spendssubstantial session time on a particular activity, then this activity isa manifestation of a particular goal. The requests of pages of theindividual classes are counted. For each session, a ranking [C.sub.i1] [less than or equal to] ...[less than or equal to] [C.sub.iN] of main learning goal(s) representedby learning activity classes is produced based on the number of requestsfor each class, which gives us some insight into the goal/task hierarchyand the importance of some of the goals and tasks. This can begeneralised Adj. 1. generalised - not biologically differentiated or adapted to a specific function or environment; "the hedgehog is a primitive and generalized mammal"generalizedbiological science, biology - the science that studies living organisms for all sessions of a learner or for groups of learners.This technique can be used in an iterative it��er��a��tive?adj.1. Characterized by or involving repetition, recurrence, reiteration, or repetitiousness.2. Grammar Frequentative.Noun 1. evaluation process. Initialclassifications might turn out too unspecific Adj. 1. unspecific - not detailed or specific; "a broad rule"; "the broad outlines of the plan"; "felt an unspecific dread"broadgeneral - applying to all or most members of a category or group; "the general public"; "general assistance"; "a general rule"; and can be refined intomore fine-granular classifications, identifying more specificactivities, tasks, and goals; thus providing more detailed andmeaningful analysis results. Behavioural Patterns and Activity Identification The goal identification is a tool on an abstract level. Oftenhowever, a closer look at interactions at a lower, fine-granular levelis necessary in order to investigate learning activities in detail. Theobjective of this mining technique is to extract behavioural interactionpatterns from the log file. Irrelevant activities--students might lookup A data search performed within a predefined table of values (array, matrix, etc.) or within a data file. other pages, even leave the system temporarily--can be discarded dis��card?v. dis��card��ed, dis��card��ing, dis��cardsv.tr.1. To throw away; reject.2. a. To throw out (a playing card) from one's hand.b. . Thefiltered sequences are candidates for sequential patterns. In order tofind out what patterns learners follow, the sequences are subjected tosome threshold control--another filter to discard too uncommon ones. Behavioural patterns encompass more than sequences: learners repeatelements, choose between options, or work on several course elements inparallel. A model of the course topology, navigation infrastructure andinteractive elements abstracted by nodes and arcs, underlies behaviouralpatterns, Figure 2(b). A behavioural pattern is an expression of alearning activity language that describes potential or actual learningas interactions with an educational multimedia system. (attendLect(1) | attendTutor(1)[.sup.+]); (attendLect(2) | attendTutor(2)[.sup.+]); ... In this example, attendLect (1) and attendTutor (1) are activities.The expression specifies that the learner can use either lectures ortutorials (the | -operator). The tutorial An instructional book or program that takes the user through a prescribed sequence of steps in order to learn a product. Contrast with documentation, which, although instructional, tends to group features and functions by category. See tutorials in this publication. might be attended repeatedly(the + -operator), before the next lesson is looked at (the; -operator).These expressions can also be represented in a graphical form, seeFigure 2(b). These behavioural patterns can be a reflection of theinstructor's intended use or an abstraction of the learner'sbehaviour. The learning activity language to describe these patterns isan integral part of the analysis model that is used to interpret miningresults. Therefore, we need to relate these behavioural patterns withthe sequential patterns extracted from the Web log. As explained inSection 4.3, activities can be associated with the transitions betweenthe nodes (URLs) reflected in the log. More advanced results can beachieved if, for example, the time spent on each activity and otherproperties are included in the evaluation (Xu, 2003). Time Series and Strategy Identification Time series are sequences of measurements over a period of time,Figure 3(a). These measurements can include results from any of themining techniques. The purpose is the detection of change in learningbehaviour, which is often a reflection of the overall learning strategyover the duration of a course. This is important for two reasons. First,change might be intended by the instructional designer, and the actualoccurrence of change needs to be verified. Second, unexpected changesneed to be detected. Time series evaluations allow the detection andconstant monitoring of student activity changes. An example for the first case is an evaluation of scaffoldingfeatures through behavioural pattern analysis. Fading fadingfading skin coloring. See Arabian fading syndrome (below). Declining in body condition, general health, activity and productivity.Arabian fading syndromegeneral health is unimpaired. use of scaffolds,features that support students in becoming self-reliant and competent ina topic, is an essential characteristic that is expected to happen in aneffective scaffolding implementation. Apart from behaviour change,changes of learning strategies have also been observed in the casestudy. Early patterns often show single-goal use, but later patternsshow a concurrent, integrated usage of different educational services.Time series of usage patterns can illustrate the evolution of studentlearning from web logs. Behavioural Styles and Learning Style Identification Learning style models address a learner's preferred way oflearning. One category of models has activities of informationprocessing information processing:see data processing. information processingAcquisition, recording, organization, retrieval, display, and dissemination of information. Today the term usually refers to computer-based operations. based on a learner's interactions with the educationalenvironment at its centre. An example is Kolb's Learning StyleInventory (Kolb, 1984). Another category is sensory sensory/sen��so��ry/ (sen��sor-e) pertaining to sensation. sen��so��ryadj.1. Of or relating to the senses or sensation.2. perception oriented o��ri��ent?n.1. Orient The countries of Asia, especially of eastern Asia.2. a. The luster characteristic of a pearl of high quality.b. A pearl having exceptional luster.3. ,related to modalities in multimedia systems. [FIGURE 3 OMITTED] Kolb, for instance, proposes a theory of experimental learning.suitable for capturing learning activities, that involves four activitydimensions: concrete experiences, reflective observation, analyticconceptualisation (artificial intelligence) conceptualisation - The collection of objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them. , and active experimentation. Kolb derives four typesof learners based on their preferred activities. Pages of the databasecourse were classified according to according toprep.1. As stated or indicated by; on the authority of: according to historians.2. In keeping with: according to instructions.3. activities, for instance, concreteexperiences: tutorial animations, or reflective observation: lecturematerial giving context and concepts. These can be associated toKolb's learner types. See Figure 3(b). Using classification yieldsa ranking of classes and, consequently, an indication of the preferredlearning style. A Case Study--Experience in a Web-Based Course Environment The mining techniques have been used extensively in a virtualcourse environment (Pahl & Donnellan, 2002). The virtual course, adatabase course taught to third-level computing computing - computer students, focuses onpractical aspects of database engineering. Various types of mediaimplemented using Web technologies facilitate different forms ofinteraction with the course content. Several learning activitiesincluding attending lectures and working in tutorials and labs aresupported through an integration of different interactive educationalservices. The lectures consist of combined audio-visual material: therecorded lecturer's speech synchronised Adj. 1. synchronised - operating in unison; "the synchronized flapping of a bird's wings"synchronizedsynchronal, synchronic, synchronous - occurring or existing at the same time or having the same period or phase; "recovery was synchronous with therapy"- with the presentation ofweb pages. The audio stream is fully controlled by the student. Thetutorial and lab services support active learning. Tutorials aresupported by various forms of animations based on HTML HTMLin full HyperText Markup LanguageMarkup language derived from SGML that is used to prepare hypertext documents. Relatively easy for nonprogrammers to master, HTML is the language used for documents on the World Wide Web. and flashtechnology--again with a high degree of control of presentation by thestudent. Lab features facilitate interaction with software tools.Various styles ranging from form fill to direct manipulation areprovided that allow students to acquire and train skills. Additionalresource centres and self-assessment features complement the coreservices The introduction to this article provides insufficient context for those unfamiliar with the subject matter.Please help [ improve the introduction] to meet Wikipedia's layout standards. You can discuss the issue on the talk page. . The integration of different educational services, which themselvescomprise a variety of interaction styles and media, enables a number ofpossible learning strategies and activities for the students. Theevaluation of the student's learning behaviour based on the actualcourse usage is therefore of paramount importance. The course has beenevaluated using freeware Software that is distributed without charge and which may be redistributed without charge by its users. However, ownership is retained by the developer who may change future releases from freeware to a paid product (feeware). See shareware, free software and public domain software. data analysers for basic statistical analysesand custom-built mining tools for the advanced aspects (Donnellan, 2002;Xu, 2003). The use of usage mining technology has benefited bothlearners and instructors alike. Web mining has been used in different styles. * In a predictive style to confirm expectations and validatedesigns. Learning styles and strategies and their change over time wereanalysed. An example is the design of lab and scaffolding usage support,which has been validated val��i��date?tr.v. val��i��dat��ed, val��i��dat��ing, val��i��dates1. To declare or make legally valid.2. To mark with an indication of official sanction.3. through predicted learning behaviour based onmining data. The scaffolding design is aimed at targeting irregular HEIR, IRREGULAR. In Louisiana, irregular heirs are those who are neither testamentary nor legal, and who have been established by law to take the succession. See Civ. Code of Lo. art. 874. learner usage indicating the need for support and feedback * In a generative style to improve the design. Theinstructor's expectations of student learning behaviour wereexpressed in a learning activity language and compared with actualbehaviour. The navigation features and the course topology have beenconstantly improved using this technique, which has resulted in anincreasingly closer match between expected (and consequently supported)behaviour and actual behaviour. * In an explanatory style Explanatory style is a psychological attribute that indicates how people explain to themselves why they experience a particular event, either positive or negative. Psychologists have identified three components in explanatory style: Personal. to understand student learning in a novelenvironment. Mining techniques have been used to clarify theunderstanding of the learner's goals and task hierarchies. Miningtechniques have been used to investigate the sequential and concurrentuse of educational features. The Web, in contrast to traditionalclassroom-based teaching, allows different educational features to beused concurrently. This analysis formed the starting point for thedesign of a multi-feature learning environment. Education-specific mining techniques--see Figures 2 and 3--havehelped to improve the instructional design and also to confirmdelivery-related decisions that were made. It has supported instructorsin providing quality instruction, and it has also helped to achieve abetter learning experience for learners. DISCUSSION AND CONCLUSIONS Customer relationship management is a central aim in the e-commerceenvironment. We suggest to make learner relationship management anobjective for e-learning. Along with evaluating and understandingcustomer needs and customer behaviour, understanding the learner andlearning processes is essential for instruction. More insight intolearner needs and behaviour through evaluation supports the instructorin optimising and improving instructional design and the learningexperience, in particular for novel and evolving learning environmentssuch as the Web. The benefits of data mining technology in discovering latentknowledge make it a convincing solution for the observation-basedevaluation of learner behaviour in feature-rich educational environmentsin particular. The comparison with classical forms of evaluation, whichare usually error prone, intrusive in��tru��sive?adj.1. Intruding or tending to intrude.2. Geology Of or relating to igneous rock that is forced while molten into cracks or between other layers of rock.3. Linguistics Epenthetic. and difficult to automate To turn a set of manual steps into an operation that goes by itself. See automation. , emphasisesthe benefits. Further improvements could be achieved by widening theevaluation basis. Mullier, Hobbs, and Moore (2002) suggest contentmining, the automated au��to��mate?v. au��to��mat��ed, au��to��mat��ing, au��to��matesv.tr.1. To convert to automatic operation: automate a factory.2. analysis of content. Moreover, combinations withclassical evaluation techniques are possible (Oliver, 2000). Interaction is central in active learning. We have embedded Inserted into. See embedded system. thislearning-oriented notion of interaction into a more technical view ofinteraction in multimedia systems, based on the hypothesis thatinteraction behaviour is a reflection of learning goals and strategies.Data mining is a technology suitable to extract different aspects of alearner's interaction with educational media from access logs.Mining results can be interpreted in analytic educational models ofinteraction and learning to further our understanding or to uselearner-specific context data in personalised learning Personalised Learning is the tailoring of pedagogy, curriculum and learning support to meet the needs and aspirations of individual learners.Personalised learning is a hot topic within the debate on education taking place in the UK at present (2006). and adaptiveeducational multimedia. Models and languages allow us to express goals and tasks aslearning concepts, to capture interaction behaviour in educationalmultimedia systems, and to relate and integrate these descriptions. Morework has to be invested into languages and models in order to integratedesign and evaluation. The presented results are a first step toward anincremental Additional or increased growth, bulk, quantity, number, or value; enlarged.Incremental cost is additional or increased cost of an item or service apart from its actual cost. methodology for instructional designers based on design andevaluation iterations that reflect evaluation results in theinstructional design. Formative evaluation is the strategy toward betterunderstanding and improved design, but only if we integrate pedagogical ped��a��gog��ic? also ped��a��gog��i��caladj.1. Of, relating to, or characteristic of pedagogy.2. Characterized by pedantic formality: a haughty, pedagogic manner. and technical concepts of activity and form a coherent analytic model,we are able to successfully improve instructional design for educationalmultimedia through Web and data mining. ACKNOWLEDGEMENTS The research described here has been supported by the Dublin CityUniversity Dublin City University (DCU) (Irish: Ollscoil Chathair Bhaile ��tha Cliath) is a university situated between Glasnevin and Whitehall on the Northside of Dublin in Ireland. Teaching and Learning Fund. The author wishes to thank Lei Xuand Dave Donnellan, who implemented most parts of the mining techniquesdescribed here. REFERENCES Agrawal, R. & Srikant, R. (1995). Mining Sequential Patterns.Proc. 11th International Conference on Data Engineering ICDE ICDE International Conference on Data EngineeringICDE International Council for Open and Distance EducationICDE International Council for Distance Education , Taipei,Taiwan. Chang, G., Healey, M.J., McHugh, J.A.M., & Wang, J.T.L. (2001).Mining the World Wide Web--An Information Search Approach. Boston:Kluwer Academic Publishers. Cooley, R., Tan, P.-N., & Srivastava, J. (1999). Discovery ofInteresting Usage Patterns from Web Data. Proc. WEBKDD'99, 163-182.London: Springer-Verlag. Dewey, J. (1916). Democracy and Education. New York New York, state, United StatesNew York,Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : MacMillan. Dix, A., Finlay, J., Abowd, G., & Beale, R. (1993).Human-Computer Interaction. London: Prentice Hall Prentice Hall is a leading educational publisher. It is an imprint of Pearson Education, Inc., based in Upper Saddle River, New Jersey, USA. Prentice Hall publishes print and digital content for the 6-12 and higher education market. HistoryIn 1913, law professor Dr. . Donnellan, D. (2002). User Session Classification Tool for theAnalysis of Web Server Logs. M.Sc. Dissertation dis��ser��ta��tion?n.A lengthy, formal treatise, especially one written by a candidate for the doctoral degree at a university; a thesis.dissertationNoun1. . Dublin City University.School of Computing. Elsom-Cook, M. (2001). Principles of Interactive Multimedia.London: McGraw-Hill. Hirami, A. (2002). The Design and Sequencing of e-LearningInteractions: A Grounded Approach. International Journal on E-Learning.1(1). Jonassen, D.H. (1994). Thinking technology: Toward a constructivist con��struc��tiv��ism?n.A movement in modern art originating in Moscow in 1920 and characterized by the use of industrial materials such as glass, sheet metal, and plastic to create nonrepresentational, often geometric objects. design model. Educational Technology, 34(4), 34-37. Kolb, D.A. (1984). Experiential ex��pe��ri��en��tial?adj.Relating to or derived from experience.ex��peri��en Learning. New Jersey: PrenticeHall. Moore, M.G. (1992). Three types of interaction. The AmericanJournal of Distance Education American Journal of Distance Education (AJDE) is an academic journal of research and scholarship in the field of distance education in Americas, with particular emphasis on the uses of Internet (e-learning, distributed learning, asynchronous learning and blended learning). , 3(2), 1-6. Mullier, D., Hobbs, D., & Moore, D. (2002). Identifying andUsing Hypermedia hypermedia:see hypertext. The use of hyperlinks, regular text, graphics, audio and video to provide an interactive, multimedia presentation. All the various elements are linked, enabling the user to move from one to another. Browsing See browse. Patterns. Journal of Educational Multimediaand Hypermedia, 11(1), 31-50. Norman, D.A. (1988). The Psychology of Everyday Things. New York:Basic Books. Norman, K.L. (1998). Collaborative interactions in support oflearning: Models, metaphors, and management. In Hazemi, R., Wilbur, S.,& Hailes, S. (eds). The Digital University: Reinventing the Academy.London: Springer-Verlag. Northrup, P. (2001). A Framework for Designing Interactivity intoWeb-based Instruction. Educational Technology, 41(2), 31-39. Ohl, T.M. (2001). An Interaction-Centric Learning Model. Journal ofEducational Multimedia and Hypermedia, 10(4), 311-332. Oliver, M. (ed.) (2000). Special Issue on Evaluation of LearningTechnology. Educational Technology & Society, 3(4). Pahl, C. & Donnellan, D. (2002). Data Mining for the Evaluationof Web-based Teaching and Learning Environments. Proc. E-Learn 2002World Conference on E-Learning in Corporate, Government, Healthcare,& Higher Education higher educationStudy beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. . AACE AACE Association for the Advancement of Computing in EducationAACE American Association of Clinical EndocrinologistsAACE American Association of Cost EngineersAACE Association for the Advancement of Cost Engineering . Ravenscroft, A., Tait, K., & Hughes, I. (1998). Beyond theMedia: Knowledge Level Interaction and Guided Integration for CBL Cblcobalamin. Systems. Computers and Education, 30 (1/2), 49-56. Sims, R. (1997). Interactive learning as "emerging"technology: A reassessment ReassessmentThe process of re-determining the value of property or land for tax purposes.Notes:Property is usually reassessed on an annual basis. You may request a "reassessment" if you disagree with your assessment. of interactive and instructional designstrategies. Australian Journal of Educational Technology, 13(1), 68-84. Weston, T.J. & Barker, L. (2001). Designing, Implementing, andEvaluating Web-Based Learning Modules for University Students.Educational Technology, 41(4), 15-22. Xu, L. (2003). Analysis of Behavioural Patterns using Web Mining.M.Sc. Dissertation. Dublin City University. School of Computing. Zaiane, O.R. (2001). Web Usage Mining for a Better Web-basedLearning Environment. Proc. Conference on Advanced Technology forEducation CATE'2001, 60-64. Zaiane, O.R. (2002). Building a Recommender Agent for e-LearningSystems. Proc. 7th International Conference on Computers in EducationICCE'02, 55-59. AACE. CLAUS PAHL, DUBLIN CITY UNIVERSITY, IRELAND E-MAIL e-mail:see electronic mail. e-mailin full electronic mailMessages and other data exchanged between individuals using computers in a network. : Claus.Pahl@dcu.ie