I think the following 6 levels of AI awareness describe the difference between a very stupid, inhuman AI and a deviously frightening opponent that seems to think like a human.
Lvl 0 - I thoughtlessly follow a routineBefore there was any such thing as AI in games, we generally just dealt with completely scripted enemies that followed set movement paths and attack rotations.
Some examples that spring to mind are the aliens in Galaga and Koopa Troopas from the first Super Mario Brothers game. In modern games, it's generally not likely to see this sort of opponent except for "inanimate" objects and environmental hazards: moving sawblades, laserbeams, etc.
[Some stealth games and RPGs do put enemies on a predetermined patrol as the most basic part of their scripting, but since they change their behavior when they detect a player, they aren't truly Lvl 0 opponent.]
Lvl 1 - I blindly pursue a single goalThe racecars in Pole Position and the Hammer Bros from Super Mario are enemies that want something: they want to get to the finish line or throw a projectile toward wherever you're standing. They don't do anything but try to complete that one goal. This is really the most basic form of AI scripting.
[The ghosts in Pacman also mindlessly chase you at first, but they pay attention to whether you've become able to kill them or not, which gives them a higher level of awareness than Lvl 1 enemies].
Lvl 2 - I think about only myselfThese are NPCs that can change their behavior based on their own current state. If an enemy heals itself when its health is low, or tries to escape combat when outnumbered, they're thinking about themselves. Same goes for any enemy that realizes they have some sort of buff or invincibility and behaves more aggressively as a result.
There are lots of NPCs in games that heal themselves, self destruct at low health, or try to run away. World of Warcraft has examples of all 3 of these mechanics.
Lvl 3 - I think about other peopleThis is the level at which some NPCs can start to seem to have some personality or even deviousness. When a character in The Sims refuses a hug from a smelly person, or a group of enemies in Starcraft focuses fire on a damaged opponent, their AI is making decisions based on the state of another entity.
Another variety of this type of AI will change between offensive and defensive modes based on how much damage they take, or build "hate" or "aggro" toward enemies that damage it the most or in certain ways. This model is now extremely common in RPGs.
[Depending on how you think of it, the ghosts in Pacman that switch between chasing you and being chased are either thinking about themselves or other people. They either realize they are vulnerable or that their enemy has become invulnerable, but in this case the effect is the same.]
Lvl 4 - I think about what other people are thinkingNow, imagine if the ghosts in Pacman could calculate that you were trying to get to one of the power pills that would place them in danger, and start running early, or hug the center of the map. If they saw that a pill was already used up in an area, they would be much more bold. Suddenly the enemies in that game would feel much more intelligent, and be much more difficult to defeat.
AI that thinks of what a player's goals are and reacts to them are the approximation of a mediocre human opponent. Chess bots are the first thing that comes to mind when I think of this kind of AI.
Also, enemies in fighting games that try to predict the player's next move and preempt it are also on this level of awareness. If the fact that I've been playing more offensively causes an enemy to play more defensively, they're making decisions based on the fact that I will probably decide to keep attacking.
[I'm not sure if this is how the AI in fighting games actually works, though]
Lvl 5 - I think about what other people are thinking about meThis level of awareness in AI is pretty rare, mostly reserved for chess bots, RTS armies, and fighting game opponents. This is essentially AI that can trick you.
If a chess bot lures your bishop out of the way with a pawn so that it can capture your queen, or an enemy in an rpg feigns death until you let your guard down, these are all tactics based on making the player think something that isn't true.
Likewise any AI that has the ability to bluff and try to scare away a player that could actually attack them falls under this same umbrella. Generally I'd only expect to see this sort of thing in poker or strategy games.
Imagine in an RPG if enemies that have detected you in stealth didn't immediately go "HUH?" and look at you, but just subtly tried to walk toward you without indicating that they'd seen you. They'd use the fact that you thought they hadn't seen you to get in close before ambushing you at the last second. This sort of AI would be incredibly scary and make players very paranoid.
Human opponents and Yomi LayersAI with an awareness level of 4 or 5 are generally the best way to prepare players to face off against human opponents, but they still aren't as dangerous as a real person. It's amazing how many more layers of deception human players are capable of, often without even realizing that we're doing it.
Check out this great article by Dave Sirlin about what happens when I start thinking about what you're thinking about what I'm thinking about what you're doing (and beyond). He refers to this as Yomi Layer 3, and apparently in competitive fighting games it's common for players to think about Yomi Layers 3, 4, and even 5.
The term "Yomi Layer" is probably new to you, but it describes a concept that should be very familiar if you've ever seen the The Princess Bride.