One of the most enjoyable and challenging things about designing videogames is that they have to succeed on many separate levels to succeed as a whole. Any bad element can be enough to sink the whole thing. This means that you have to trust your teammates and be suspicious of yourself. No matter how experienced you may be, there's a designer out there with twice your experience who has managed to screw up a good game.
I've seen beautiful software with bad gameplay, and amazing games held up by some pretty awful software. One of my favorite games is such a bad piece of software that after a year of playing it, I eventually, sadly, gave up on it. I still daydream that one day I'll reinstall it and the developers will have fixed all the crash bugs and awful lag and added a keybinding ui that works. I miss that game! No amount of good gameplay will save you if your software is so unusable or unstable that your biggest fan has to uninstall it.
Depending on what kind of game you're making, one aspect or another may be the most noticeable. People might not have much time to really notice the interface flaws in an 8 hour DS game, but that same UI might start driving people crazy and causing cancellations if it were inserted into an online game that people play day after day for years. Seemingly insignificant frustrations can become dealbreakers when compounded by time.
When you're working on a feature or some new content for your game, stop and take a moment to think how it will feel after 100 hours of accumulated play time. One extra click to loot a corpse can really stack up, when a player has fought 10,000 enemies, on 10 different characters, over 3 years. The extra 4 seconds it takes the game to load, or the small framerate drop due to bad implementation can start to drive people crazy.
People don't always quit games over big things. Big things ironically annoy enough people that they often get fixed fairly quickly. That one annoying detail that never gets enough complaints to be fixed can really get under someone's skin, and make them finally give up. Have you ever seen a couple start a big argument over something that you didn't even notice? Once something has a history of frustrating someone, their tolerance for it will become even lower than it initially was, and they'll begin to find fault in things that fresh set of eyes would never even notice.
As a designer, you generally can't hope to play your game even a tenth as much as your hardcore players. If you're testing your game and something seems mildly unresponsive or inconvenient to you, assume that one of your players somewhere is becoming so enraged by it that they're about to throw your game in the trash.