Silver Ore isn't even worth the time unless you're dead set on getting it yourself. You're better off buying it from the AH, where it's always cheap, much to my woe (as a miner), or just getting it from the NPC. If you're going into goldsmithing, 500 gil an ore is something you should love.
When I'm mining, I throw them away. They just take up a spot where Darksteel/Mythril/Platinum/Iron/Zinc/etc could be.