Smith clarified in an email to ESPN's Bob Holtzman that he has never met Fisher but said he stands by his opinion.
"It's my opinion and I played with guys that be scared of a challenge in the locker room we call them Jag= just another guy," he wrote in the email.
The Chiefs went with recently signed Jah Reid at right tackle instead of Fisher, the No. 1 overall pick in the 2013 draft.
Fisher is active and in uniform for the game, but Reid was taking snaps with the starters in pregame warm-ups.
Fisher was listed as questionable to play in Sunday's game on the Chiefs' final injury report of the week. He missed the final three preseason games because of the injury, but early in the week the Chiefs said he would start at right tackle. He participated in all of the week's practice sessions.