Yes. Workers are entitled to 11 hours of continuous rest every day (12 hours for young workers), which yours will not be getting if, for instance, they start at midday and do not finish until the small hours. Moreover, if they work late on several nights a week, you might be caught by the regulations on night working (see question 19).