utilitai

# NOTES This is a collection of notes I took while learning about Utility theory for Game AI. I have left them unedited, so they may not be useful, but the resources I used were integral to the creation of this library. ## Resources http://www.gameaipro.com/GameAIPro/GameAIPro_Chapter09_An_Introduction_to_Utility_Theory.pdf https://www.gdcvault.com/play/1012410/Improving-AI-Decision-Modeling-Through https://www.gdcvault.com/play/1021848/Building-a-Better-Centaur-AI ## Terminology ### Reasoner * has a list of possible choices * play with ball * build unit * select weapon * etc * each choice has a list of considerations * evaluate one aspect of the situation * considerations generate appraisals * appraisals inform our final decision ### Consideration * encapsulates one aspect of a larger decision * distance * cost * selection history * benefit * etc. * parameterized for easy customization * for _this_ decision for _this_ entity * example interface has two main functions: `load: (node: TDataNode) => void` `evaluate: (context: TContext) => Appraisal` ### DataNode * XML, JSON, or equivalent data that contains parameterization * tool-generated * part of larger AI specification ### Context * contains all information the AI needs to make a decision * provides abstraction layer between AI and game * if well implemented, can be ported game to game ### Appraisal * generated by evaluate (?) * drives final decision * common techniques * boolean (all return TRUE) * highest score * weight based random * START AS SIMPLE AS POSSIBLE, EXTEND ONLY WHEN NECESSARY * Simple Utility-Based Appraisals: * each appraisal contains two components: * *Base Score*: utility value of decision * *Veto*: boolean that allows each consideration to prevent selecting associated choice * calculating a total utility for a choice: * if any Veto is false, utility is 0 * else, add all base scores together * Example: Weapon Selection * "tuning consideration" provides a base score * always returns data-specified values, regardless of situation * Pistol = 3, Sniper = 5 * "range consideration" add utility or veto as needed * Pistol has higher value at short range, snipers at long * "inertia consideration" adds utility to current choice * "random noise consideration" has a random base score * "ammo consideration" checks if we have ammo * "indoors consideration" prevents grenade use indoors * select weapon with best score * Extending Appraisals: * Multipliers * replace Veto with a "Final Multiplier" * add base scores together, then multiply by each of the final multipliers * allows for smooth scaling * Multi-Utility Appraisals: * Add a Priority attribute to appraisal * When combining appraisals, take max Priority * Only consider choices with max priority * allows you to conditionally consider small sets of options * ex: forcing melee weapon at short range, ranged at long range ## Tips Inertia: * a solution to the "oscillation" problem (AI rapidly switching between actions) * Possible solutions: * * add weight to any action that is already engaged * * cooldowns - apply a strong weight once a decision has been made that can drop off over time * * stall the decision making system (time based or until action is completed) Compensation Factor: * problem: as you multiply normalized values, the total drops * (1 * 0.9 * 0.9 * 0.9) < (1 * 0.9) * punished for having more considerations * Compensation Factor can be used to somewhat self balance * ModificationFactor = 1 - (1/numConsiderations) * MakeUpValue = (1 - Score) * ModificationFactor * FinalConsiderationScore = Score + (MakeUpValue * Score) * example: * 1 - (1/6) = 0.833 * (1 - 0.9) * 0.833 = 0.083 * 0.9 + (0.083 * 0.9) = 0.975 Input Implementation * Decision "Context" * Decision identifier (enum) => what am I trying to do? * Link to intelligence controller object => who is asking? * Link to Content Data with parameters => what do you need? * Optional link to context object => who is this happening to? * Input example: Target Health ```ts const considerationTargetHealth = (context) => { const eid = context.getTargetEntity(); const entity = gameState.getEntity(eid); if (!entity) return 0; return entity.currentHealth / entity.maxHealth; } ``` * Consideration maps inputs into decisions * parameterized Inputs example: Distance to Target ```ts const considerationTargetDistance = (context, consideration) => { const eid = context.getTargetEntity(); if (!eid) return 0; const targetPosition = gameState.getEntity(eid); const position = context.getPosition(); const distance = calcDistance(targetPosition, position); const rangeMin = consideration.GetParameter(PARAM_RANGE_MIN); const rangeMax = consideration.GetParameter(PARAM_RANGE_MAX); return Math.clamp((distance - rangeMin) / (rangeMax - rangeMin), 0, 1); } ``` Use pre-defined, parameterized curves Decisions * linked to code functions, but completely separate from AI * Decision Parameters: * some decisions can have params, some require them * emote [emote name] * run script [script name] * options * tags Decision Score Evaluators (DSE) * Represent a decision process * evaluate inputs via considerations * score * if selected, make decision * "why am I doing what?" * Two Types: * Non-Skill DSE * directly mapped to single decision * Skill DSE * not inherently associated with 1 skill * assigned later * Common components * name * description * considerations * weight * optional parameters * considerations[] => score * weight = final score * example DSE: ```ts // bonus = weight + momentum + other bonuses = max possible score const DSE = (context: DecisionContext, bonus: number, min: number) => { const finalScore = bonus; for (const consideration of m_considerations) { if (0 > finalScore || finalScore < min) break; const score = consideration.score(context, consideration.parameters); const response = consideration.computeResponseCurve(score); finalScore *= Math.Clamp(response, 0, 1); } return finalScore; } Making a decision ```ts ScoreAllDecisions (decisions, decisionContext) { let cutoff = 0; for (const decision of Decisions) { const bonus = decision.getContext().getBonusFactor(decisionContext); if (bonus < cutoff) continue; const dse: DecisionScoreEvaluator = decision.getDSE(); decision.score = dse.score(decision.getContext, bonus, cutoff); if (decision.score > cutoff) cutoff = decision.score; } } ``` Decision Maker Packages * supplemental DM's that are _in addition to_ what is defined in core intelligence * situational packages are "handed out" by events, objects, map * processed alongside the core DM * removed when no longer needed * example: * villager goes to tavern, which has a tripwire * gives "tavern behaviors" to DM package * move to interior POIs * higher priority wave, look at, etc * "exit tavern" has a leave tripwire that removes behaviors Influence Maps * representation of the world in terms of "influence" that agents project into it * easy to aggregate influence information from multiple sources * processed once for other agents to use * eliminates n^2 problems / redundant calculations * use cases: * information about a location * my location * target location * finding a location * where to move * where to target spell * general information about the area * does a concentration exist? * how big is it? * Components: * Knowledge Representation * processing of location / threat information * storage of information in the world * basics of retrieval * Modular Construction * treat each layer as atomic component * algorithms for shaping data * assemble "recipes" * Personal Interest Template: * centered on self * starts at 1 * falls off to 0 at max range * multiple scores to "cull" less interesting information Priority Boost / Cut tags * adjust final score of DSE * situation or event-driven * leader points out preferred target * event-specified priority target * example priority layer: * boost extreme = 2.0 * boost large = 1.5 * boost medium = 1.25 * boost small = 1.1 * cut small = 0.9 * cut medium = 0.75 * cut large = 0.5 * cut extreme = 0.25 The key to understanding Utility theory is to understand the relationship between the input and the output, and being able to describe that resulting curve. Think of it as a "conversion process" * linear * quadratic * piecewise linear curves ## Designing Curves m = max x = value w = x / m ### Linear: y = x / m * consistent relationship between value and utility ### Quadratic (exponential): y = (x / m) ** k * extreme variance at high and low ends of value * * large k value has little impact for low x values * * conversely, low k value (0 to 1) has greater impact on low x values ### Logistic: y = 1 / 1 + e**-x * largest rate of change in the center of the input ### Logit: y = (log)e(x/(1-x)) * large rate of change at beginning and end ### Piecewise: hand-tune a plot of 2d points fully customizable * ex: (hunger, utility) => (0, 1), (15, 1), (25, 0.75), (40, 0.3), (60, 0.05), (80, 0), (100, 0) * we can use linear, quadratic, or any type curve within each of the ranges * * for 25 to 60 we could use a quadratic and use linear for the others EXAMPLE FUNCTIONS: Logistic: `U(w) = 1 - (1 / 1 + (Math.E * 2)**-(w*12)+6)` Logit: `U(w) = (Math.log * Math.E * (w / 1 - w) + 5) / 10` [number of allies] [strength of allies] [number of enemies] [strength of enemies] \ / \ / [allied strength] [enemy strength] \ / \ / \ / [threat ratio]