back to article Montezuma's Revenge can finally be laid to rest as Uber AI researchers crack the classic game

Montezuma’s Revenge, the classic Atari platform game, has finally been fully solved by machine learning, researchers from Uber AI Labs claim. Reinforcement learning (RL) eggheads have been fascinated with the old 1980s game for a while. It features an Indiana Jones-like character named Panama Joe, who goes around exploring …

  1. graeme leggett Silver badge

    Confused.

    Perhaps I'm missing something but the method reads a lot like 'play the game a lot and remember which bits work'? Which is very human but not very inspired.

    In my defence, it's first thing in the morning for me.

    1. Shadow Systems

      Re: Confused.

      Don't feel bad, you're not alone in the confusion. It sounded to me that it amounted to "Did $X work? If True then remember $X else forget $x & try $y instead" repeated at random until the bot eventually figures out the sequence of $x's that completes the game. Ok so they tried a new algorythm to do it, but it's still just lipstick on the pig. Maze solving bots have been around for ages. Unlike the maze solving bots of yore though, you're letting this one pick a random starting point inside the maze. That's cheating. You have to figure out the path to get to that point inside the maze before you can continue from there, you can't simply teleport into the maze & say "I'm starting from here. Don't like it, too bad." WE don't get to teleport to a random starting point, so neither should the bot. You start from the entrance like everyone else. If you die en route then you start over from the beginning like the rest of us. And that's for a static maze that never changes. You want to teleport in at random? Fine, but we get to randomize the maze if you do, every time you do, so your teleportation results in you never knowing if you're solving the same maze as last time. Don't like it? Too bad.

      Now shut up, go back to start, & find the damned cheese the right way.

      If you'll excuse me I need caffeine & a nap. =-)p

      1. Teiwaz

        Re: Confused.

        Don't feel bad, you're not alone in the confusion. It sounded to me that it amounted to "Did $X work? If True then remember $X else forget $x & try $y instead" repeated at random until the bot eventually figures out the sequence of $x's that completes the game.

        Ooh! I wrote something a bit like that into a Battleships game (in Pascal - it was a course project) - does that mean I was doing A.I. back in 1990?

        1. Hans Neeson-Bumpsadese Silver badge

          Re: Confused.

          Ooh! I wrote something a bit like that into a Battleships game (in Pascal - it was a course project) - does that mean I was doing A.I. back in 1990?

          Given the amount of stuff that gets branded as AI nowadays, I think anyone who has ever wielded an algorithm in anger can claim to be an AI engineer.

    2. caffeine addict

      Re: Confused.

      That confused me too.

      If it's "play game. if that worked replay what you did last time" then this isn't AI, it's brute force.

      1. Dan 55 Silver badge

        Re: Confused.

        I do hope Uber won't brute force navigation routes in the same way.

        1. This post has been deleted by its author

        2. Sssss

          Re: Dan's, Uber comment.

          "Look there's a river, I should take a short cut through there. I'll figure out if bridges are important afterwards".

      2. DaLo

        Re: Confused.

        You're right it's not AI ... it's called marketing.

    3. Anonymous Coward
      Anonymous Coward

      Re: Confused.

      Yes it is exactly "play the game a lot and remember which bits work". Brute forcing a game like this is far easier than brute forcing chess, which is how all computers play chess and no humans do.

      I wonder if they were using a simulator able to run at a much higher speed than the actual game, or just played thousands of instances of the game in parallel?

    4. Aristotles slow and dimwitted horse

      Re: Confused.

      Yup me too.

      To be fair though, isn't that exactly what we do as humans though? We keep trying different approaches, discard the ones that don't work, and remember the ones that do? Anything else is the definition of madness.

      Unless it's that final Giant Fireclaw in Horizon Zero Dawn : The Frozen Wilds. I've just not been able to beat that bugger - so have given up :-(

      1. Khaptain Silver badge

        Re: Confused.

        "To be fair though, isn't that exactly what we do as humans though? We keep trying different approaches, discard the ones that don't work, and remember the ones that do? "

        The difference being that we understand very quickly the objectives and if necessary will return to what we previously considered as bad and we don't just randomly move form point to point, trying every possible combination.

        We also "perceive" pleasure or frustration which can also influence our gaming, sometimes that is a bad thing though.

        An algorithm is just an algorithm and brute force methods are not exactly "intelligent"... they are "brute"....statistically they will eventually arrive at the end.

        If their algorithm is any good they should be able to apply it to any game, which I very much doubt. I would rather the programmer researched cancer or heart disease solutions rather then spending time solving limited puzzle games. If their only goal is the eventual monetizing of taxi routes than that is truly sad, too much intelligence going to waste.....

        1. Hans Neeson-Bumpsadese Silver badge

          Re: Confused.

          If their only goal is the eventual monetizing of taxi routes than that is truly sad, too much intelligence going to waste.....

          Their eventual goal probably involves applying this for driverless vehicles, so this also leads to putting people out of a job....which is even sadder

      2. Jason Bloomberg Silver badge

        Re: Confused.

        To be fair though, isn't that exactly what we do as humans though? We keep trying different approaches, discard the ones that don't work, and remember the ones that do?

        And we also learn by observing what the results are from other people's attempts or experiences, and advice we are given.

        How else would we know not to go into the basement of a creepy house when the power goes out or to decline the red T-shirt on a space mission?

        I'll deem AI a success when one humanoid robot goes into a room full of nut-jobs with baseball bats and one robot comes out rubbing its palms together.

      3. Anonymous Coward
        Anonymous Coward

        @Aristotles slow and dimwitted horse

        We keep trying different approaches, discard the ones that don't work, and remember the ones that do?

        But we aren't trying stuff totally at random, unless it is a really poorly designed game. There should be something that gives you a hint of which path is better than another. If you just blunder left/right/up/down at random until something kills you or you get points, that's a game not fit for anyone over 4.

      4. Kane
        Terminator

        Re: Confused.

        "Unless it's that final Giant Fireclaw in Horizon Zero Dawn : The Frozen Wilds. I've just not been able to beat that bugger - so have given up :-("

        Use freeze arrows to weaken it's armour, then either the hard point arrows (with as high a damage modifier(s) on the bow that you can muster), or sticky bombs.

        And keep moving!

        Pic related, cos you know, robots!

    5. phuzz Silver badge

      Re: Confused.

      "play the game a lot and remember which bits work?"

      There is another part mentioned in the article which goes "strategy X worked on the last level, I'll try that again first", which is a small step closer to how humans learn, and would put it about level with the reasoning capabilities of some of the users I deal with.

  2. Anonymous Coward
    Anonymous Coward

    Still..

    .. this game was created in the 80s and has taken until now to fully analyse with machine learning, using far more computer power that what it was designed for.

    I take my hat off for the original developers.

    1. Anonymous Coward
      Anonymous Coward

      Re: Still..

      It isn't as if it has taken this long due to the difficulty. It is just that hardly anyone cares about these games anymore, let alone cares about them enough to try to make a computer play them well. I assume the head researcher is of the right age to have played this as a kid and still have a soft spot for it, and decided to use it as a test.

      If it had been paid the amount of attention chess has, it would have been cracked 25 years ago.

  3. macjules
    Holmes

    " ... using far more computer power that what it was designed for."

    Oh dear.

    I take it that being Uber the RL algorithm is not going to be used to play 1980's games but to compute the best way for an automated vehicle to get from X to Y? I can just see the lines of stationary automated EV's rapidly running out of power while the "A.I." tries to calculate how to get across London* in the rush hour while avoiding the "popup" roadworks.

    (* - always presuming that that company is ever allowed to operate in London again after the 15 month respite.)

  4. Harry Kiri

    I salute the people in white coats

    for eventually coming up with a custom guessing algorithm to solve a custom problem.

  5. Anonymous Coward
    Anonymous Coward

    Scanning the article, I formed the notion that the game was actually Custer’s Revenge.

    Somehow I found that more in character for Uber.

    1. Andalou

      Me too. But I think that must be THQ Nordic's downtime project.

  6. jms222

    Go-Explore great

    Go-Explore sounds great for teaching a bot to drive. Just keep going with eventual feedback based on the number of people mown down.

  7. Kane
    Coat

    Not impressed...

    ...and I won't be until I see a demonstration of AI/ML completing Bloodborne.

    That would be impressive.

    Mine's the one with the blood vials in the pocket, and the +10 upgraded Saw Cleaver on the back.

    1. Kane

      Re: Not impressed...

      Hmm. 2 thumbs down. Not a fan of the Saw Cleaver then?

  8. RyokuMas
    Big Brother

    "the path to success is not obvious"

    I'm guessing that once this technology is perfected, Uber will be using it in the real world to better avoid officials in london while still finding genuine suckerspassengers.

  9. DropBear

    Sounds like brute force indeed

    Okay, you know what - I'd like to see an "AI player" that learns to complete Thunderbirds (yes, exactly the ones you are thinking about, complete with pool animation in its intro), _without_ having a custom path-finding algorithm hand-coded inside it first. Let's see it "evolve" one...

    1. Hans Neeson-Bumpsadese Silver badge

      Re: Sounds like brute force indeed

      Have an upvote for the Thunderbirds reference - a most excellent game that I'd forgotten about, so thanks for the reminder

  10. Lee D Silver badge

    I'm still not seeing AI.

    As in... at all.

    It's basically given a simplified graph (these keys in these rooms in this order) and path-finds down the tree to work out how to get there.

    It's human-written heuristics guiding a very-limited-scope "AI" which wanders aimlessly (nothing wrong with that... A* is basically a random walk at times) and then scores itself based on a human heuristic for those "cells".

    It's not AI, learning, or anything close to that at all. If this is really "best of breed" in terms of AI, then it shows what I've known all along - we don't have AI and won't have for a long time to come.

    1. Mark 85

      I'm not either. This seems more like a variation of the old game strategy of "save early, safe often", such that when you screw up, you go back to a good part and try again but differently. Not really AI, as far as I can tell. But then AI is the buzz phrase of the year at this point.

      1. asdf

        Another 1980s game it should try

        >"save early, safe often",

        Or in the case of Rogue and its permadeath saves take many NetBSD VM snapshots. Yes I cheated and even then it wasn't trivial to beat and required some trial and error on my part too. Those mobs in the bottom few levels are best avoided at all costs. When an AI can consistently beat that game then I will be seriously impressed (one of the first games with procedurally generated levels).

        1. asdf

          Re: Another 1980s game it should try

          Permadeath autosaves that is.

      2. eoink

        Nothing "Intelligent" going on here, just TS problem with a limited set?

        Discuss.

  11. Flakk

    Are They Renting It Out?

    I wouldn't mind it finishing "Halo" for me on Legendary difficulty.

  12. TRT Silver badge

    Of course the real world is very different...

    Such as not having three lives, for example. I mean, all well and good having robots running around a warehouse learning stuff, but without cognitive threat avoidance it's going to be expensive having to replace all the flattened scutters whilst Kryton learns not to put the crates of lead shielding on shelving levels greater than zero.

  13. Anonymous Coward
    Anonymous Coward

    I not Robot

    “Say you had a warehouse with robots running around, this method could help teach the robots how to navigate if the layout of the warehouse stays the same.”

    If you had a warehouse with a fixed layout, you wouldn't need AI to control your robots, and the control systems can be very primitive, with corresponding robustness.

    As for calling these learning systems "AI", I'll believe it's an AI when you need Dr. Susan Calvin, or Powell and Donovan to diagnose problems when something goes wrong

  14. diver_dave

    Other challenges

    How about giving it a crack at Starquake.

    Very different every time. Goal objects change every game as do key locations....

    I love my Speccie EMU.....

    Dave

    1. Anonymous Coward
      Anonymous Coward

      Re: Other challenges

      I was thinking about other games which are variable. Lords Of Midnight is one which came to mind - IIRC what you might find at any given location depended on when you got there. It would be interesting to see the algorithm (I hesitate To call it an AI engine) learn a path through the game and try to repeat it, without the delays associated with the learning process, and get a different result.

  15. Locky

    Level1

    Right, give them ET and see how it copes with that abomination

  16. Anonymous Coward
    Anonymous Coward

    Would you like to play a game?

    What happens when they try to play Global Thermonuclear War?

    1. Elfoad Regfoad

      Re: Would you like to play a game?

      A strange game.The only winning move is not to play. How about a nice game of chess?

  17. pyite42

    Possible to play this game on Linux?

    I haven't played this game since the 80's! I would love to give it a try and see how well it has aged.

    1. Anonymous Coward
      Anonymous Coward

      Re: Possible to play this game on Linux?

      Don't know about Linux, but I think many have unwillingly played it on a toilet.

      1. Teiwaz

        Played Global Thermonuclear War?

        Don't know about Linux, but I think many have unwillingly played it on a toilet.

        For a moment there that sounded familiar, then I recalled it was just the aftermath of a particularly unsuccessful chilli experiment.

        'Don't have the sprouts!'

    2. Flakk

      Re: Possible to play this game on Linux?

      I haven't played this game since the 80's! I would love to give it a try and see how well it has aged.

      STELLAAAA!

  18. Jim 59

    Now try it on Roland in Time.

  19. Anonymous Coward
    Anonymous Coward

    This only is suitable for certain game play. In reality, the highest score at the moment might detract from the overall completion of the path, or the overall score in other games or reality. But as they say, for certain environments it is more suitable.

    It is only a bit of intelligence, something that mimics no-brainer intelligence. But in raslirgz a human might be accessing different paths, with an intelligence of understanding the environment and what to do about it. With enough intelligence, it might even run right through the game with little to no wastage first time. But then you get into levels of reading the game designs intent and waying up negatives and positive elements and probability, and understanding the game designers intent and theory of self actualselg, motivations and intended theory of self they perceive of the game players. This includes deliberate deviations and tricks to tool the players thinking and multi layering if those. Really not no-brainers.

    There is a solution to this all.

    Wayne Morellini

    1. DailyLlama

      Did you really sign your name at the bottom of a comment posted by "Anonymous Coward"?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like