The 1967 track may be one of The Kinks most well known songs now, but the 76-year-old writer initially just wanted it to be a private musical moment for his family.
He recalled: "I got my family together- my sister was over from Australia and there was Jackie my niece and a couple of my nephews -and I played them the acetate of it about 20 times.
"I said, 'That's for us and I don't want it to come out.'
"It was so important to me I didn't want it to come out. Shows I'm not a very good businessman. I just wanted it to be ours."
But Ray admitted he's "pleased" by the reception the song got and the way people have connected to its themes.
He added to MOJO magazine: "I'm pleased that people connected to it in the right way. And regardless of the nationality thing, it's two young lovers looking at the sunset seeing their future."
The 'Sunny Afternoon' hitmaker admitted one of the best things about his career is when people acknowledge the songs he's written - even if they don't do it for the right reasons.
He said: "It's incredible when someone says, 'Did you write that mister? I'd love to have written that.'
"The reward for writing is not how much money it makes, the sales, it's recognition from other people, that it resonates.
"But it's part of me. Just imagine if I hadn't written 'Waterloo Sunset', what an empty place my life would be...
"When I lived in New Orleans, I lived in Treme. My next door neighbour was a guy called June. He said 'Hey, Ray, you the guy that wrote 'Waterloo Sunset'?' I said, 'Yeah, how'd you know that?' He ssid, 'I know these things.' He said he had a gig at the Funky Butt. 'I need 20 bucks to get my horn there. Can you lend me 20bucks?' I gave him 20 bucks. 'Thanks man, I'll put your name on the door.'
"I get to the Funky Butt, and there's no June, no gig. But he knew 'Waterloo Sunset'."