Building Artificially Intelligent Learning Games

Games and Simulations in Online Learning: Research and Development Frameworks By David Gibson, Clark Aldrich, Marc Prensky
Chapter XIV Building Artificially Intelligent Learning Games
Richard Van Eck,
book
online, earlier version?

He asks:

1. What mechanisms exist in other fields that can be used to present content within a game in a way that is compatible with the game and game principles?
2. What mechanisms exist in other fields that can support the principles of scaffolding, question asking, and problem solving?
3. How must these mechanisms be modified according to the principles outlined here and other theories or approaches?
4. How, assuming we can answer the first three, can we make sure that intelligent learning games are extensible to multiple problems and domains, and ensure that any content expert can generate content for these games without “sucking the fun out” of them?

He believes that one partial solution to the first of these questions lies in an area of study in cognitive psychology and instructional design called pedagogical agents. Pedagogical agents are typically used in computer-based instructional environments where learners interact with a computer-based character to get advice, feedback, or instruction.

As an example, he refers to the infamous Microsoft talking paper clip “Clippy” as a pedagogical agent. He believes that such agents can deliver information without interrupting “flow”

I believe that this is related to misconceptions of endogenous fantasy which I have discussed before. Yes, having to read tracts of information in a game breaks flow, but not because it breaks the fantasy, rather it breaks flow because it is not honest, because it breaks the authentic problem solving process of the game.

When players are playing World of Warcraft, they will often break out to third party help sites such as Thottbot to find where particular items are more likely to drop. Flow is not broken, the interrogation of Thottbot is an integral part of the problem solving process. The break in fantasy is of no great significance to the player's attention to the problem being solved. The gameplay is what is of primary importance, the fantasy is just icing on the cake. Flow has more relationship to

• the ability to match challenge to ability,
• suitable interim rewards,
• an authentic and relevant challenge
• and a supportive community which can facilitate a ZPD.

The talking paperclip is an example of exactly what is not required, when problem solving in MS Office, the user wants quick and efficient access to the necessary information to solve the current challenge which may be writing a letter, they do not want to enter into a social relationship with a talking paperclip. The paperclip breaks flow, though the fantasy of a paperclip may be relevant to the endogenous fantasy of a paper based office, it is an unauthentic impediment to the problem solving process, the gameplay of writing a letter.

So what mechanisms can be used to present content within a game in a way that is compatible with the game and game principles? The instructional content itself should be consistent with with the game goals.

Physics sims/microworlds

Thanks for Physics Phun
(posted on February 22, 2008 by Daniel Livingstone)

Which describes 2 new physics sims: Phun and Crayon Physics

for similar stuff see:
linerider
soda constructor
Armadillo run


see also
http://curvebank.calstatela.edu/home/home.htm
http://www.ct4me.net/math_manipulatives.htm
http://www.fi.uu.nl/wisweb/en/welcome.html
http://objectsevolve.wikispaces.com/
http://schoolgamemaker.rupert.id.au/samples3/

Virtual Labor Lost

Virtual Labor Lost
The failure of a highly anticipated, multiplayer game shows the limits of academic virtual worlds.
Technology Review
Wednesday, December 05, 2007
By Erica Naone

Erica reports on the failure of Arden, the World of Shakespeare an educational MMORPG and research testbed made by Edward Castronova. It was financed by a $250,000 MacArthur Foundation grant. Castranova says the project failed because it wasn't fun to play.

Mainstream MMORPG's have been used for research into Dunbar's number and probably more opportunities will be found.

Maybe the lesson is that with the web, its no longer so important to create content as to find and re-purpose existing content. Google now indexes over 14 billion pages.

The changed business model is demonstrated by the open source community. With so much freely available information, there is no longer an advantage in restricting intellectual property, smart businesses share it and make money from other services. Similarly researchers who wait for publication in refereed journals may find that research has moved on before they can publish.

So if you are looking for an educational game or a research testbed, the first choice should be using or re-purposing what's already freely available.

Quest Atlantis

The Design and use of simulation computer games in education Ed Shelton & Wiley
Chapter 7 "The Quest Atlantis Project"
Barab et al.
http://www.sensepublishers.com/catalog/files/9789087901554.pdf

The paper questions whether the values of computer games are the ones we want for children, white male protagonists, sexualised women, tokenised minorities and violent themes. Even those with benign or empowering values are not grounded in the real world or critical in orientation.

"The space available for youths to express their agency has been reduced from several square miles to a mere computer screen, yet even that domain is appropriated by commerce... difficult to imagine spaces not touched in some way corporatised signs" I disagree, this is a bit dated, in the hundreds of square miles of World of Warcraft, I dont see a single sign.

"Quest Atlantis sits at the intersection of education, enertainment and social action."

Seems to be a series of projects set in the matrix of a game like dates in a pudding.

• a 3d Multi user virtual environment
• inquiry learning quests and unit plans
• a storyline through video, novella and comic
• community of participants

They can "build their personal persona through their homepage functions" may be modelled on Facebook or Myspace.

"Completing quests requires that children participate in academically meaningful activities , either in the real world ... or through simulation".

Data from over 3300 questers, a quest on plant and animal cells showed significant learning of conceptual understandings. A history quest resulted in a deep appreciation of how content related to their life and multiple perspectives. QA students had deeper character insights than those doing worksheets.

Students, particularly boys wrote more, resulting in "gender equitable" outcomes.

Players sent 50,000 lines of in game chat and 1500 in game emails.

Playing Saving Adryanee

Saving Adryanee is a learning game by Dr Sebastian Loh of Southern Illinois University Collaboraty for Interactive Learning Research (CILR). It is a playable module for Neverwinter Nights made with the Aurora Toolset.

I was lucky to have the opportunity to play this game which is not yet released as it is still under development. I won't give too much detail because it is not the final version and I wouldn't want to bias any trials.

Adryanee is sick and to save her, you need to discover some nutrition facts and collect some medicines. It is a quest game like World of Warcraft where information and items are 'dropped' by NPC's (Non Player Characters).

The game took me around 2 hours but I could now re-run it in 10 minutes. Embedded in the game are 3 nutrition based facts associated with a general idea of poor nutrition causing diseases. The game could be played without processing this information because it was not actually necessary to recall the information. Once you had 'learnt' the fact, your avatar 'knew' the fact and this altered NPC interactions. There was never a dialogue choice that depended on correct recall or application of a fact. There was also a theme of problem solving being research based and not about superstitious learning. Facts should be tested.

Nevertheless my attention was focused on these facts because I expected that they would be relevant in solving the game.

The learning content was well integrated into the game, in the sense of endogenous fantasy (Malone and Lepper)

In terms of playability and engagement, I found the game too hard and not rewarding. Detailed searching of NPC's and objects is not my idea of fun though it may well suit other playing styles. It should be noted that it is a game under development and that it will no doubt be modified as a result of further playability trials. What was absent was the ability to match the level of difficulty to the players ability. Also missing were interim rewards.

Consider the learning on these criteria:
Declarative knowledge
I can recall all three disorders and their cures because I was engaged but recall was not necessary to play the game. My recall is enhanced because I already knew these 3 facts. 3 facts in 2 hours is not a high information density.

Procedural knowledge
At no stage was I required to use the declarative knowledge. If, later in the game, I was required to correctly diagnose and treat NPC's, the knowledge would cease to be 'inert' knowledge and be better retained. Though my avatar assembled and evaluated data with a scientific method, the player was not required to evaluate. It was scientific method by example not by practice.

Problem solving
Have my problem solving skills been improved? What do I take away that might allow me to do evidence based research. Might my skills at solving infectious disease or forensic medicine problems be enhanced? I was shown by example, causality, evidence, evaluation of evidence but I was not ever problem solving in this domain. I was doing a lot of problem solving in the game domain. I knew I needed to get into the inn and I knew what items I needed. I was second guessing the game designer and drawing on my experience in playing similar games.

Watch this space. Its early days for learning games. The Aurora Toolset has a lot of potential for the cost effective creation of content. Purpose built modules like this can have inbuilt audit trails, see Loh, Designing Online Games Assessment as “Information Trails”

A big thank you to Dr Sebastian Loh for making this game available despite power outages and deep snow at that time. A dwarf walked up to me and gave me advice, it took a sentence of dialogue or two to be sure it wasnt a NPC, It was Sebastian, thanks!

Problem solving - creating runnable mental models

Game creation is gaining recognition as a valuable learning activity. I have justified it in the past on 3 grounds

• transferable cognitive skills,
• metacogitive skills and
• affective benefits

and my attention for transferable cognitive skills has mainly been on near transfer,

• Cartesian coordinates
• negative number
• position, speed, acceleration
• and many more like this
  

I would like to argue the case for generalised problem solving skills in the visual domains like mathematics and physics through improved competence at creating and using runnable mental models.

Bloom' Taxonomy (note 1) is not particularly helpful for understanding higher order thinking and problem solving in visual thinking so I have already had a try at describing problem solving in visual domains as the creation and running of mental models.

Last year I described how one could solve problems like eg. the forces in structures, from first principles with a bit of prior knowledge by building, validating and running a mental model

There is a bit of literature on mental models, Betrancourt & Chassot refer to a "runnable mental model" in Mayer, R. E. (1989). Models for understanding. Review of Educational Research but I can't download that. Jonassen 1 2 refers to runnable mental models:

He says:
(1) Mental models are internal representations.
(2) Language is the key to understanding mental models; i.e.. they are linguistically mediated.
(3) Mental models can be represented as networks of concepts.
(4) The meanings for the concepts are embedded in their relationships to other concepts.
(5) The social meaning of concepts is derived from the intersection of different individuals' mental models.
These assumptions, we believe, are probably necessary but not sufficient for defining mental models....

Generally, mental models are thought to consist of

• an awareness of the structural components of the system and their descriptions and functions,
• knowledge of the structural interrelatedness of those components,
• a causal model describing and predicting the performance of the system (often formalized by production rules),
• and a runnable model of how the system functions

Jonassen's mental might be a bit different to my concept, mine are visual rather than linguistically mediated. Jonassen also states that mental models are multi-modal so the meaning of linguistically mediated is unclear. Jonnasen also seems to lump together mental models that are a community mental model that is socially negotiated with the construction of a problem space inside a problem solver's head.

I like the words in Dunn quoting Jennifer Wiley: students’ “active construction of a runnable mental model” significantly improves their comprehension of any dynamic system.

To give another example of creating, validating and running mental models, consider my manual transmission car which has a noise. Assume I have a vague recollection of the function of the clutch to break the drive train and remove forces from the gearbox. I produce two visual images of engine-clutch-gearbox-wheels and engine-gearbox-clutch-wheels as possible mental models of the car. To validate them I run them, test their output for what I know of the behaviour of cars. Only the first model is consistent with double declutching (remember having to do that? You are old). I can now use my validated model to diagnose my noise, the noise is present, stationary in neutral , but not with the clutch depressed. Run the model, only the gearbox input shaft meets these conditions.

My belief is that there are generalised problem solving and higher order thinking skills that relate to the ability to build complex and robust mental models and then interrogate or run them. Good problem solvers are good at building and running mental models. These skills can be exercised when programming, particularly in the syntax-free iconic languages such as Scratch, Etoys and GameMaker, also when playing problem solving games. The higher order thinking is the debugging where you compare the behaviour of your program and the mental model of your program. Good learning environments keep learners in a tight cycle of test-implement-debug.

Care should always be exercised when talking about generalised higher order thinking skills, one can make sweeping claims without ever defining what higher order thinking is. Pea & Kurland (ON THE COGNITIVE EFFECTS OF LEARNING COMPUTER PROGRAMMING) criticised similar claims about Logo: whether "spontaneous experience with a powerful symbolic system will have beneficial cognitive consequences, especially for higher order cognitive skills. Similar arguments have been offered in centuries past for mathematics, logic, writing systems, and Latin" That is why I think it is important to have a clear understanding of what higher order thinking is.

I'm thinking about the mental model you have of a computer program as you write the program and the debugging process in the context of cognitive conflict or cognitive dissonance. How helpful is it to view a mental model of a program in the dimensions of the Event Indexing Model - time, space, protagonist, causality, and intentionality? For example, think about event driven programming vs linear, with causality indexing for event driven vs temporal for linear.

Finally, you add the magic ingredient of games: a relevant and authentic challenge, the right tools and a collaborative environment which encourages peer tutoring, flow and the ZPD.


Note 1, Reeves "Kyllonen and Shute (1989) have proposed a taxonomy that represents the spectrum of internal states with which cognitive psychologists are concerned. Their taxonomy begins with simple propositions (e.g., stating that Japan sells more electronic products than any other nation), proceeding through schema, rules, general rules, skills, general skills, automatic skills, and finally, mental models (e.g., analyzing the potential of a trade war between Japan and the United States based on an analysis of balance of trade trends). The latter type of knowledge seems particularly important because mental models are the basis for generalizable problem-solving abilities (Halford, 1993)."


more to come.. work in progress

Integrating Computer Technology into the Classroom, Morrison & Lowther

Integrating Computer Technology into the Classroom
Third Edition
By
Gary R. Morrison
and Deborah L. Lowther
Chapter 1 Rethinking Computer and Instruction
http://www.prenhall.com/morrison/

"We have yet to see any large scale gains attributed to the infusion of this latest technology"

"We should use the computer as a tool rather than delivery mechanism"

The NTeQ model involvers 10 steps for creating a lesson plan. The student is actively engaged in the learning process, assumes the role of researcher and becomes technologically competent. A bit like discovery learning or problem based learning?

"Problem-based learning (PBL) is a student-centered instructional strategy of in which students collaboratively solve problems and reflect on their experiences." "Discovery Learning is a method of inquiry-based instruction and is considered a constructivist based approach to education." Wikipedia

Specify objectives
Match computer functions
Specify the problem
Data manipulation
Results presentation
Activities during computer use
" before "
" after "
Supporting activities
Assessment

(I am suspicious of anything with exactly 10 steps)

Chapter 2, NTeQ a research based model
The No Child Left Behind act requires that an instructional strategy must be proven effective with rigorous research. This is often interpreted as standardised testing which carries a risk of biasing education to instructivism and away from constructivism.

Chapter 2 asserts that NTeQ model is effective. The "factory model' of education is critised and the NTeQ model is effectively identified as a constructivist model "student centred approach", computers as a tool to solve problems rather than deliver instruction.

Quotes a study Authentic Intellectual Work and Standardized Tests: Conflict or Coexistence? 1/2001. Fred M. Newmann, Anthony S. Bryk, and Jenny Nagaoka which compares authentic original work and standardised testing. In a study of 2000 classroom assignments and standardised scores for 5000 students, they found that authentic intellectual activities were 20% more effective than the national average whereas a memorisation approach was 22-25% less effective than national average.

Ross et al, found a number of gains, assessed by students, teachers and parents, though not a rigorous study. Student achievement was assessed in terms of writing performance on a prompted essay. Results significantly favored the Laptop group on all evaluation dimensions—Organization, Ideas, Style, and Conventions.

For an alternative view see Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching . Kirschner Sweller & Clark

Gamemaker competition and GameCreator in C++

From Learning Games:

Gamemaker competition and GameCreator in C++

Posted on February 4, 2008 by Daniel Livingstone

I know quite a few educators are using Mark Overmar’s Gamemaker. His YoYo games site has a new competition - to create games with an Egyptian theme. Top prize of $1000. More here.

Meanwhile, I noticed a short while ago that the new edition of Microsoft Visual C++ Express Edition (the free download one) includes The Game Creators game development kit. This is essentially the same library as used in DarkBASIC, but with a C++ API - and significantly improved performance. No I haven’t had a chance to play with, and no I’m not likely to in the near future. But if you are looking for an introductory C++ game creation toolkit, it has to warrant a closer look.

Simulations and Games in e-Learning

e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, 2nd Edition
Ch 15 Simulations and Games in e-Learning
Clark & Mayer
http://www.researchandmarkets.com/reports/c74748

They repeat the (unproven) assertion that kids brains are differently hard wired by playing games but do not necessarily endorse it. They make reference to beliefs about the role of fantasy which may not be well founded. Cognitive Load theory is acknowledged "a simulation or game will be successful to the extent that it does not overload working memory"

Situations where games did not advance learning are quoted (Rieber, Hays). They conclude that game goals and learning goals should align.

Quoted Randel who reviewed 67 experiments, 22 favoured games, 3 conventional instruction and 38 no difference.

Games and Simulations Principle 1: Match Game Types to Learning Goals.
Games and Simulations Principle 2: Make Learning Essential to Progress.
Games and Simulations Principle 3: Build in Guidance.
Games and Simulations Principle 4: Promote Reflection on Correct Responses.
Games and Simulations Principle 5: Manage Complexity.

The Peripatos could not have looked like that

The Design and use of simulation computer games in education Ed Shelton & Wiley
Chapter 6 "The Peripatos could not have looked like that"
Moeller, Cootey & McAllister
http://www.sensepublishers.com/catalog/files/9789087901554.pdf

Several game modding environments are mentioned as appropriate for building educational games, Half Life 2, Unreal Tournament & Neverwinter Nights.

Aristotle's Assassins was an educational game built with Neverwinter Nights at the Learning Games Initiative at Utah State University and the University of Arizona. It was designed to teach politics philosophy and music of ancient Greece. The design team consisted of a faculty member, a graduate student and two undergraduates.

They discovered that the team was learning a lot in developing this game. This unexpected learning is examined in the theoretical framework of Toward implementing distributed scaffolding: Helping students learn science from design Sadhana Puntambekar , Janet L. Kolodner and also Activity Theory

They conclude "The student design of games and simulations is a highly educational process, and the steps they take out during the design process can be helpfully interpreted using activity theory as an analytical framework".

They refer to similar findings, Learning Through Game Modding MAGY SEIF EL-NASR and BRIAN K SMITH

Learning Through Game Modding
MAGY SEIF EL-NASR and BRIAN K SMITH

They argue for the value of game making:
"During the design process, skills such as analysis, synthesis, evaluation, and revision, must be used, providing opportunities for learning content and metacognitive skills such as planning and monitoring. Students can receive ongoing feedback from peers and experts while constructing working artifacts".

They mention studies using other programming environments, Conway (Alice), Ingalls (Squeak), Repenning (Agentsheets), Resnick (?), Smith (Kidsim). References could have been made to Game Maker, Click and Play, Stagecast etc but were not.

The three modding tools they look at are Warcraft 3, Webdriver and Unreal 2.5 Engine.

"Working with Web Driver...(for) a better understanding of 3D geometry, and vector mathematics... Unreal Tournament ... (for) architecture design, texturing and sound design, lighting design and landscaping ... 3D geometry"

See student projects of Penn State College of Information Sciences and Technology you can see the design documents and then view the completed games. You can see a movie of Dungeon Girl which "included over 6020 lines of code, 5 characters, 24 weapon models, 5 original music files, and 80 sound effects and dialogue files" Watch the video, it is an example how with a relevant and authentic challenge and the right tools, students can achieve excellence.

"Through these two classes, we gained better insight on the use of game modding as a tool to promote learning. We believe that there are several skills and concepts that students learn by engaging in game design/modification, including the following:

• Software Development and Design
o team work
o building critiques and reflections on other’s work
o project scheduling
o project management
o iterations and refinement
o prototyping
• Programming Concepts
o threading and event-based programming
o Object-oriented programming
o Component-based development
o Software patterns
• Artistic Concepts
o Lighting
o Architecture design
o Character design
• Game Concepts
o Game design
o Game mechanics
o Balancing game aesthetics and game play"

They conclude "evidence ... encourages the use of game modding in classes to promote learning of several subjects and concepts. .. We believe that using game modding motivated students to learn and allowed them to apply and visualize the utility and application of the concepts".

My Conclusion
The findings of both studies (Moeller et al & El-Nasr et al) should not come as a surprise Compare EL-NASR and SMITH's list with a similar list . Read about Harel's research and a project inspired by it.