Fruit, vegetable and berry are more nutritional terms than biological ones. Vegetable in nutrition can be taken to mean any part of a plant used for nutritive purposes (and some government agencies go farther, defining ketchup, a product made from the fruit of the tomato plant and extraneous ingredients like sugar, as a vegetable).
Fruits are fruiting bodies, not necessarily reproductive (think slime mold, banana, strawberry) with a subcategory "berries" for non-reproducitve fruiting bodies. Yeah, tis hard to conceptualize but the banana is a berry while the blueberry is a fruit.
Most peppers are fruits, as is the tomato, but the potato is what? It is capable of reproducing the plant... Oh my. Could it be a berry? How about walnuts, pecans, peanuts? Are they fruits of some sort? Will a new plant arise from them? Or are they just nutritive seeds?
This is the sort of quandary one runs into when concentrating too much on names and taxonomy in general. But it can increase awareness by Zen shock. Think fruits and vegetables and then consider the pineapple. Imagine the sound of two hands clapping then the sound of one hand clapping...