I'm a big guy, so the Note 3 fits me very well. I have it in a leather wallet case and it's replaced my wallet, so I still only carry one bulky thing in my pocket.
The really great thing about the Note 3 is the battery. It has a really long battery life because of the size, and it charges very quickly because it has that new MicroUSB connector that charges much faster. N3 is backwards compatible to the older MicroUSB as well but it won't charge as quickly.
My wife got an S4 around the same time I upgraded. She just doesn't have the hands for the N3 and doesn't have use for the stylus. In fact, I don't use the stylus on the N3 just because I stopped using them years ago and I don't feel like going back. They're both great handsets.
mac, if you have the hands for it, the bigger screen size is awesome. I love using the Navigation, for instance.