Gotcha! Scarlett Johansson got quite the scare on Thursday, April 30, when her Avengers: Age of Ultron costar Chris Evans snuck up behind her on The Ellen DeGeneres Show.

DeGeneres, who has a history of scaring celebs with funny pranks on her show, was chatting with the Lucy actress, 30, about her "very delicate" costars in Marvel's latest superhero flick. They had just started to talk about Evans, 33, when he crept stealthily onto the set. 

"I think this is our fifth movie," Johansson (aka Black Widow) told her host. "We've been working together since we were like...17, I think? I know him very well. Too well, I'm sure."

At that, Evans jumped behind her, hit her chair, and yelled her name, causing the blonde stunner to nearly jump out of her seat. (See her priceless reaction for yourself in the video above!) 

Once she recovered, Evans joined her in a game of "Avenger? I Hardly Know Her," where DeGeneres read statements about the stars, and they had to guess whether the statements were true or false. Fittingly, almost all of the rounds about Evans featured shirtless screenshots of the Captain America hunk (including a nearly naked still from the rom-com What's Your Number?). 

Watch the game below (and try not to get too distracted by Evans' abs)!