No not at all
They are probably fruit flies (tiny brown flies) and they actually aid the composting process.
They are annoying to us as they swarm out when the lid is open.
Dont worry - always try to bury the food under paper, and try and keep your food sealed in a container before its put into the wormery, as most flies lay their eggs on the food waste - before it goes into the wormery - where the eggs hatch
Ronnie