Don’t come between him and Duke of Cambridge. Prince William was  “passive aggressive” to an actor who “flirted” with Kate Middleton, according to the actor himself, comedian Jack Whitehall.

In a recent interview on The Jonathan Ross Show, Whitehall recalled the Duke of Cambridge‘s reaction to a 2014 stand-up routine he did at the Royal Variety Performance, where he revealed that he and Kate attended the same English private school, Marlborough College. During the routine, Whitehall explained that he never spoke to Kate while he was at the school, given that she’s six years older than him, but that her from becoming his “first crush.”

“We were actually at school together. This is true. We were at the same school,” he told Kate, who was sitting with William in the royal box.“I was a bit younger than you, but this is the first time I’ve seen you since school, so hello. I don’t know whether it’s appropriate given the circumstances, but you were probably my first ever crush.”

When he saw William’s reaction to him saying that Kate was his “first crush,” Whitehall said, “OK, definitely inappropriate. You Highness is giving me a very sharp look.”

In his interview on The Jonathan Ross Show, Whitehall revealed that William came up to him after his routine and was “very nice” but “passive aggressive” toward him, given his comments about his wife. “I flirted a little because I said she was one of my first crushes at school,” Whitehall said. “And then in the line-up William came over and he was very funny. Because he was very nice, but very passive-aggressive, ‘Oh so you were flirting with my missus? Oh no, no, that was very, very funny’. That’s him going, ‘I could have you beheaded.’”

Though Whitehall also described Kate as “the one that got away,” he confessed that they would’ve likely never dated. “She was a lot older than me,” he said on The Jonathan Ross Show. “She was in sixth form ‒ it was never going to happen, I was in a head brace!”

Kate and William married in April 2011 after 10 years of dating. The couple share three kids: Prince George, 7, Princess Charlotte, 5, and Prince Louis, 2.