Well, if an author is writing some bullshit that he/she made up or their work isn’t based on fact, it ceases to be non-fiction. If I know of an author who wrote something that is patently untrue in a supposed non-fiction book, I’d definitely pass on reading it.
On rare occasions, I’ll stop reading a “non-fiction” book if the sources are obviously shady or unsubstantiated. An example would be last year’s book by Jesse Ventura, They Killed Our President: 63 Reasons to Believe There Was a Conspiracy to Assassinate JFK, which used a bunch of random conspiracy-minded websites or blogs as sources to back up Ventura’s claims. When I saw the sources that Ventura cited, I was stunned — it was actually embarrassing to see someone believe that the sources cited were even in the same neighborhood as reputable.