The actors watched a playback of the hilarious gag during an episode of the daytime talk show, which airs on Monday, May 2. Evans, 34, came up with the idea to hide in Olsen's dressing room for the prank.
Olsen, 27, walked into the bathroom after being told that there were free QVC products. Instead, Evans popped out from the corner. She screamed and said: "Oh my God! Now I'm sweating!"
Evans loves to scare his costars. He previously ambushed Scarlett Johansson during The Ellen DeGeneres Show in April 2015.
"I remembered it afterwards. I was like, 'Well, when Scarlett was on last year for Avengers, he scared her. Why didn't I think of that?'" Olsen said on Monday.
Evans added: "How could you resist? This is the best part about your show that you endorse this type of behavior. I'm really, really addicted to it."
DeGeneres couldn't let Evans have all the fun, though. Watch the host and Olsen get even in the video above!
Sign up now for the Us Weekly newsletter to get breaking celebrity news, hot pics and more delivered straight to your inbox!