My fish accidentally ate some of my gravel!!!?

Are you sure he didn't spit it back out? I always see my fish picking up pieces of gravel but they spit them back out when they realize it is not food.