I've been reading a book debunking various word myths (
Word Myths, by David Wilton), and the myth about "nitty gritty" is covered there. The phrase originated in the 1950s. It's just a bit of rhyme -- nitty was probably added just to rhyme with gritty, as was done with similar sounding phrases like "namby pamby."
ETA: There's a really thorough debunking of the "rule of thumb" myth by Wilton here:
http://www.wordorigins.org/index.php/more/489/
erwins