As
@Wizard mentioned, a rod that's to stiff will cause this a lot Jimmy.
Lipless cranks are known for losing fish and a lot of it is because the bait is small but fairly heavy. The weight of it is all concentrated in a small area, so when a bass shakes it's head, the weight of the lure works like a lever to their advantage.
I typically use a medium action rod with these baits, vs. a medium heavy. And be sure to use line with some stretch to it, not fluoro or braid. Because of its stretchy properties, Berkley Big Game works well.