IMO, they should. If you are an elite prospect, and good enough to enter the NBA that early, college isn't worth the risk.
1) People say education is important, which is true, but if you know you are going to the NBA anyways you aren't gonna care about class during that 1 year. Also, you can always go back to school at any age, while there's a limited amount of time to be in the NBA.
2) The 1 year learning from a college coach can help a player's growth, but they'd be better off learning from NBA coaches or playing in the G-league if necessary. They'd also have more time to work on their game if they didn't have to worry about school.
3) Some players need the money to support their family, so getting an NBA contract would secure that.
4)An injury in college could derail a player's draft stock or even entire career.
5)Colleges giving scholarships to these elite prospects takes away scholarships from other kids who would stay multiple years at the school. It's also not good for some college teams to have completely different teams every year.