There is no hard and fast rule on this. Obviously when / if traffic and or orders really start to taper off it is very concerning.
But is it a branding issue? Or just a traffic / marketing issue? Neither one of them is good, but these tend to be more of a ‘traffic / marketing’ issue. I would advise you spend your time and money there first.