Sure, olive oil can be used in baking. But not for brownies. Have you ever tasted olive oil? Have you ever tasted brownies? They're not exactly on the same flavor wavelength. Technically, olive oil is considered a vegetable oil, as the broad category takes in oils from any plant source - nuts, seeds, legumes, or in the case of olives and corn, fruits. (In fact, at this moment, we can't think of any vegetable oil that actually comes from a vegetable....)
But vegetable oil specified in brownie recipes generally means canola, corn, safflower, or sunflower oils, because they are essentially flavorless. There are some quite bland, highly-processed olive oils on the market - especially "light olive oil," which is not light in any dietary sense, but has been so thoroughly filtered that it has lost most of its color and fragrance. If you're bound and determined to use olive oil, you can try a light olive oil. But anything with more flavor is going to affect your brownies, and not in a good way.
used in baking, olive oil is generally chosen for neutral or savory projects, such as pizza, focaccia, olive bread, etc.