You are correct, as an attempt by a state to tax non-residents may well violate the commerce clause.
Complete Auto Transit, Inc. v. Brady, 430 U.S. 274 (1977) held that a state tax of non-residents must meet four requirements to pass commerce clause challenge:
- Substantial Nexus: A sufficient connection between the taxpayer and the state.
- Fair Apportionment: State cannot tax more than its fair share of taxpayer’s income.
- No discrimination: State cannot treat out-of-state and in-state taxpayers differently.
- Related to services: Tax must be fairly related to services provided to the taxpayer by the state.
The weak point of a state trying to tax non-residents is the lack of a nexus between the taxpayer and the state and the absence of any real services provided by the state to the taxpayer. The theory behind allowing a government to
steal from tax citizens is that the citizens make use of the benefits of the public services provided by the state, such as roads, courts, police, and fire departments and should therefore carry a part of the burden of financing these services through taxes.
The taxes would not violate the limits on apportionment or discrimination since the state can say, "Hey, we tax the non-citizens at the same rate," for example. The idea California can tax people who don't live in the state is a bit absurd. Why can't Florida tax New York residents a ton of money since all the old and infirm New Yorkers move to Florida and clog up their highways? Because the New York residents have no connection to the state and Florida provides no services to the New Yorkers until the move to or visit Florida.
Things like hotel taxes, car rental taxes, etc. are "justified" by the fact the visitors get to benefit from the state-funded services like roads, police, etc. while in the state and should pay their fair share. However, taxing people living in another state is just not going to fly.