Jenn, I don't really know all the details, but I think it's a bit of a personal decision as to what to do. If it is just an hour or two of time missed, you may just want to put up with it. Or you could try to get make up time for the time you missed. If there's a breach of the agreement, you could file a contempt motion, but they're hard to succeed at.
As nitrocellulose points out, often the agreement is not specific enough to enforce through the courts and you may need to get it changed so that it is. If there's a solution that would resolve this sort of continual problem - for instance, alternating rather than sharing holidays - you may with to go to court to get the agreement changed anyhow.
In making your decision, you'd also need to consider the motivation of your ex in doing this - is it control, revenge, etc, or simply a lack of consideration.
You'd also need to consider legal fees. Even for a little thing like this, it can get quite expensive.