I've not done a side by side, but I have tried just about every type that is available in my area.
I find that some smoke smells a bit different, but I have not noticed a big flavor difference. The exception to that rule is mesquite that has a unique smell and flavor. I usually have alder on hand for smoking fish, hickory for pork, oak as a general purpose, and mesquite for beef. The hopper usually contains a bit of each except for mesquite, which I remove any left over pellets.
I keep the pellets in 5 gallon buckets to keep them dry and add an approximate amount that I think I will need for a cook. Next cook gets any left overs plus the new load of pellets.