I think it does matter. Going to church on Sundays is kind of fundamental for Christians!
Which makes my own position rather hypocritical, since racing on Sundays prevents my husband from going to church with our two younger children (Kevin goes along but sets up the Traidcraft stall and helps with tea rather than going to Sunday school, and Ivor won't go and isn't welcome anyway because the vicar's wife gets twitchy when he mooches around the graveyard on his own, so I stay at home with him anyway). On the other hand, he goes to church every Wednesday and could go to the early service on Sundays if he would get out of his pit in time.
For some, there's the whole Eric Liddle type issue of whether or not it's OK to compete on a Sunday at all.
I think people like Neyo who adhere firmly to their beliefs in the face of the massive temptation provided by races are to be respected. Taking my more practical approach is easier, but the path of true discipleship is a narrow one.
And what about people like Sassie, who are priests and therefore can't opt out of Sunday obligations?