You can. But this is why seasoning is commonly added to the flour. But to add a nice crust (example, pork chop), it's double or triple dredged. But none of that will stick well without the first application of flour. Try it and you'll likely find everything simply comes off like a shell into the oil. All because it can't stick to the meat. Especially as the moisture in the meat starts blowing it off from the steam during cooking.