Minni, of course you haven't! Keep on at the swimming lessons, perhaps some one to one with video analysis? If you can see how you actually swim as opposed to how you think you swim,you'll see where you need to improve.
So many Pirates have gone from almost non swimmers to Ironman inside a year, so it's never too late or impossible. You just need the desire to do it to make it happen.