Hi Janet
Move your wormery to break the route that the ants are taking to get to the wormery.
Ants aren't generally a problem, but you dont want them nesting, as this can cause problems, and they are often attracted to sweet smells from the decaying fruit etc.
If they still persist get some vaseline, and smear a line of it around the base of the wormery, this will prevent the ants getting up into the wormery
Dont use ant powder
Ronnie