Level 1
I haven't completed 4 but both are good. Yakuza 3 has two places you can visit whereas Yakuza 4 has one place you can explore (with the sewers, underground mall, rooftops).
The story in Yakuza 4 is much more darker and grittier than Yakuza 3. People can get annoyed with the orphanage parts in Yakuza 3 (I thought it was good in the first playthrough). One you complete the games you can explore and finish off a few substories. In both games the text dialogue can put you off as there is a lot of it (R1 + X for it to go quicker) and the cutscenes can bore you if you do not know what's going on (I'm alright with cutscenes, go to options and turn the event skip to ON).
In Yakuza 4 you play as 3 new characters along with Kazuma Kiryu. Each have a different fight style.
You can read my review for Yakuza 4 (and 3 but it's short) on the Amazon site, it goes into detail at some of the aspects of both game. Here's the link:
Level 3
I've just bought Yakuza 3 as it was going cheap. I've been looking at this on and off now and I wasn't quite sure if I would like it but I took the gamble and got it. Haven't tried it yet but from what I've read from the reviews, it comes highly recommended. If I like this one, I'll probably get Yakuza 4 as well.
Level 4
I like Yakuza 4 a little more. But just like Yakuza 3, the beginning is a bit slow. Takes a while until it really kicks in.
But my favorite in the Yakuza series is still Yakuza 2 for the PS2, I think it still got the best story. I would be very pleased if SEGA decided to release Yakuza 1&2 HD for the PS3! 
Website ©2013 Sony Computer Entertainment Europe
All content, game titles, trade names and/or trade dress, trademarks, artwork and associated imagery are trademarks and/or copyright material of their respective owners. All rights reserved. [more info]
