The chief of staff to embattled Mayor Rob Ford was escorted by security from city hall premises Thursday as allegations the mayor had been caught on videotape smoking crack cocaine continued to swirl.

Mark Towhey said he did not resign but his departure had not come as a shock to him.

"The mayor and I spoke about it this afternoon," Towhey said as reporters trailed him through the underground parking lot.

Towhey refused to elaborate on the conversation or say what advice he had given Ford about the alleged cellphone video.

"My conversations with the mayor are between the mayor and me," Towhey said. "My advice to the mayor is my advice to the mayor."

Last week, both the American-based website Gawker.com and the Toronto Star reported they had "seen but not obtained" a video made by a west-end drug dealer who was shopping it around for six figures.

Gawker has been trying to raise $200,000 to buy and post the video, reaching more than $135,000 by Thursday afternoon.

The website did not respond Thursday to a request for comment on its "crackstarter campaign."

Neither of the reports about the video has been independently verified and the Star itself said it could not vouch for its authenticity.

Ford has said little about the allegations, beyond calling them "ridiculous" and suggesting the Star was out to get him.

Towhey's departure comes a day after the Toronto District Catholic School Board announced it had dropped Ford as volunteer coach of his favourite high school football team, the Don Bosco Eagles.

Ford, who allegedly referred to the players disparagingly in the video, has long cited the team as an example of his selfless dedication to others.

The Catholic board made no reference to the crack cocaine allegations, saying only it had decided a new direction was needed.

Also on Wednesday, Ford's brother, Coun. Doug Ford, gave a statement in which he lashed out at the media and defended his brother.

"Rob is telling me these stories are untrue, that these accusations are ridiculous," Ford said. "And I believe him."

The councillor said his brother has stayed silent on advice of his family and lawyers, and would let the media know if and when he had anything to say.

Police have only said they were monitoring the situation.

For more from the Canadian Press, click here or scroll down The Tyee's main page.