I added the included shims and it fixed the problem. Now they're easier to clip in and out. Should have figured that out when I needed to put so much pressure on them to get them to clip in. Live and learn.
Are you aware that you don't just push down and lift up to engage/disengage? You are supposed to twist. Also, depending on the way you install the cleat it takes more or less twist to disengage.